lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sebastin <sebasmt...@gmail.com>
Subject Re: Java Heap Space -Out Of Memory Error
Date Wed, 05 Sep 2007 06:12:15 GMT

Hi testn,
           here is my index details:
                            Index fields :5 fields
                            Store Fileds:10 fields
           

Index code:              

  contents=new StringBuilder().append(compCallingPartyNumber).append("
").append(compCalledPartyNumber).append(" ").append(compImsiNumber).append("
").append(callType).toString();


records=new StringBuilder().append(compCallingPartyNumber).append("
").append(compCalledPartyNumber).append(" ").append(compchargDur).append("
").append(compTimeSc).append(" ").append(compImsiNumber).append("
").append(outgoingRoute).append(" ").append(incomingRoute).append("
").append(cgiLocation).toString();


                               Document document = new Document();
                                    document.add(new Field("contents",
                                                           contents,
                                                           Field.Store.NO,
                                                          
Field.Index.TOKENIZED));
                                    document.add(new Field("fil", filen,
                                                           Field.Store.NO,
                                                          
Field.Index.TOKENIZED));
                                    document.add(new Field("records",
records,
                                                           Field.Store.YES,
                                                           Field.Index.NO));
                                    document.add(new Field("dateSc", dateSc,
                                                           Field.Store.YES,
                                                          
Field.Index.TOKENIZED));
                                    
                                    indexWriter.addDocument(document);

inputs for the document:

compCallingPartyNumber="9840836588";
compCalledPartyNumber="9840861114";
compImsiNumber="984510005469874";
callType="1";
compChargDur="98456";
compTimeSc="984";
outgoingRoute="i987j";
incomingRoute="poi09";
cgiLocation="dft1234567";

here is my search code:

                                         Directory indexDir2 = 
                      FSDirectory.getDirectory(indexSourceDir02,false);
                                         IndexReader indexSource2 =
IndexReader.open(indexDir2);
                                       Directory indexDir3 = 
                                            
FSDirectory.getDirectory(indexSourceDir03,false);
                                         IndexReader indexSource3 =
IndexReader.open(indexDir3);
                                         Directory indexDir4 = 
                                            
FSDirectory.getDirectory(indexSourceDir04,false);
                                         IndexReader indexSource4 =
IndexReader.open(indexDir4); 
                                        
           
                                         
        IndexReader[] readArray = {indexSource2,indexSource3,indexSource4};
        //merged reader
        IndexReader mergedReader = new MultiReader(readArray);
        IndexSearcher is = new IndexSearcher(mergedReader);
        
                                
                                 QueryParser parser = 
                                     new QueryParser("contents" ,new
StandardAnalyzer());
                                 
                                 
                                String searchQuery= 
                                       new
StringBuffer().append(inputNo).append(" AND dateSc:["
).append(fromDate).append(" TO ").append(toDate).append("]").append("
").append("AND").append(" ").append(callTyp).toString();
                                   
                                 
                                 
                                 Query callDetailquery =
parser.parse(searchQuery);
                                 
                                 hits = is.search(callDetailquery); 




                                             
                                       
                                          
         

testn wrote:
> 
> Can you provide more info about your index? How many documents, fields and
> what is the average document length?
> 
> 
> Sebastin wrote:
>> 
>> Hi testn,
>>            i index the dateSc as 070904(2007/09/04) format.i am not using
>> any timestamp here.how can we effectively reopen the IndexSearcher  for
>> an hour and save the memory because my index gets updated every minute.
>> 
>> testn wrote:
>>> 
>>> Check out Wiki for more information at
>>> http://wiki.apache.org/jakarta-lucene/LargeScaleDateRangeProcessing
>>> 
>>> 
>>> 
>>> Sebastin wrote:
>>>> 
>>>> Hi All,
>>>>        i used to search 3 Lucene Index store of size 6 GB,10 GB,10 GB
>>>> of records using MultiReader class.
>>>> 
>>>> here is the following code snippet:
>>>> 
>>>> 
>>>>      
>>>>                                          Directory indexDir2 = 
>>>>                       FSDirectory.getDirectory(indexSourceDir02,false);
>>>>                                          IndexReader indexSource2 =
>>>> IndexReader.open(indexDir2);
>>>>                                        Directory indexDir3 = 
>>>>                                             
>>>> FSDirectory.getDirectory(indexSourceDir03,false);
>>>>                                          IndexReader indexSource3 =
>>>> IndexReader.open(indexDir3);
>>>>                                          Directory indexDir4 = 
>>>>                                             
>>>> FSDirectory.getDirectory(indexSourceDir04,false);
>>>>                                          IndexReader indexSource4 =
>>>> IndexReader.open(indexDir4); 
>>>>                                         
>>>>            
>>>>                                          
>>>>         IndexReader[] readArray =
>>>> {indexSource2,indexSource3,indexSource4};
>>>>         //merged reader
>>>>         IndexReader mergedReader = new MultiReader(readArray);
>>>>         IndexSearcher is = new IndexSearcher(mergedReader);
>>>>         
>>>>                                 
>>>>                                  QueryParser parser = 
>>>>                                      new QueryParser("contents" ,new
>>>> StandardAnalyzer());
>>>>                                  
>>>>                                  
>>>>                                 String searchQuery= 
>>>>                                        new
>>>> StringBuffer().append(inputNo).append(" AND dateSc:["
>>>> ).append(fromDate).append(" TO ").append(toDate).append("]").append("
>>>> ").append("AND").append(" ").append(callTyp).toString();
>>>>                                    
>>>>                                  
>>>>                                  
>>>>                                  Query callDetailquery =
>>>> parser.parse(searchQuery);
>>>>                                  
>>>>                                  hits = is.search(callDetailquery); 
>>>> 
>>>> 
>>>> it takes 300 MB of RAM for every search and it is very very slow is
>>>> there any other way to control the Memory and to make search faster.i
>>>> use SINGLETON  to use the IndexSearcher as a one time used object for
>>>> all the instances.
>>>> 
>>> 
>>> 
>> 
>> 
> 
> 

-- 
View this message in context: http://www.nabble.com/Java-Heap-Space--Out-Of-Memory-Error-tf4376803.html#a12492218
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message