lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From testn <te...@doramail.com>
Subject Re: Java Heap Space -Out Of Memory Error
Date Thu, 13 Sep 2007 11:12:50 GMT

What is the error message? Probably Mike, Erick or Yonik can help you better
on this since I'm no one in index area.

Sebastin wrote:
> 
> HI testn,
>              1.I optimize the Large Indexes of size 10 GB using Luke.it
> optimize all the content into a single CFS file and it generates
> segments.gen and segments_8 file when i search the item it shows an error
> that segments file is not there.could you help me in this 
> 
> testn wrote:
>> 
>> 1. You can close the searcher once you're done. If you want to reopen the
>> index, you can close and reopen only the updated 3 readers and keep the 2
>> old indexreaders and reuse it. It should reduce the time to reopen it.
>> 2. Make sure that you optimize it every once in a while
>> 3. You might consider separating indices in separated storage and use
>> ParallelReader
>> 
>> 
>> 
>> Sebastin wrote:
>>> 
>>> The problem in my pplication are as follows:
>>>                  1.I am not able to see the updated records in my index
>>> store because i instantiate 
>>> IndexReader and IndexSearcher class once that is in the first
>>> search.further searches use the same IndexReaders(5 Directories) and
>>> IndexSearcher with different queries.
>>> 
>>>                 2.My search is very very slow First 2 Directories of
>>> size 10 GB each which are having old index records and no update in that
>>> remaining 3 Diretories are updated every second.
>>> 
>>>                 3.i am Indexing 20 million records per day so the Index
>>> store gets growing and it makes search very very slower.
>>>  
>>>                4.I am using searcherOne class as the global application
>>> helper class ,with the scope as APPLICATION it consists of one
>>> IndexReader and IndexSearcher get set method which will hold the
>>> IndexReader and IndexSearcher object after the First Search.it is used
>>> for all other searches.
>>> 
>>>               5.I am using Lucene 2.2.0 version, in a WEB Application
>>> which index 15 fields per document and Index 5 Fieds,store 10 Fields.i
>>> am not using any sort in my query.for a single query upto the maximum it
>>> fetches 600 records from the index store(5 direcories)    
>>>                 
>>> 
>>> hossman wrote:
>>>> 
>>>> 
>>>> : I set IndexSearcher as the application Object after the first search.
>>>> 	...
>>>> : how can i reconstruct the new IndexSearcher for every hour to see the
>>>> : updated records .
>>>> 
>>>> i'm confused ... my understanding based on the comments you made below 
>>>> (in an earlier message) was that you already *were* constructing a new  
>>>> IndexSearcher once an hour -- but every time you do that, your memory 
>>>> usage grows, and and that sometimes you got OOM Errors.
>>>> 
>>>> if that's not what you said, then i think you need to explain, in
>>>> detail, 
>>>> in one message, exactly what your problem is.  And don't assume we 
>>>> understand anything -- tell us *EVERYTHING* (like, for example, what
>>>> the 
>>>> word "crore" means, how "searcherOne" is implemented, and the answer to 
>>>> the specfic question i asked in my last message: does your application, 
>>>> contain anywhere in it, any code that will close anything
>>>> (IndexSearchers 
>>>> or IndexReaders) ?
>>>> 
>>>> 
>>>> : > : I use StandardAnalyzer.the records daily ranges from 5 crore to
6
>>>> crore.
>>>> : > for
>>>> : > : every second i am updating my Index. i instantiate IndexSearcher
>>>> object
>>>> : > one
>>>> : > : time for all the searches. for an hour can i see the updated
>>>> records in
>>>> : > the
>>>> : > : indexstore by reinstantiating IndexSearcher object.but the
>>>> problem when
>>>> : > i
>>>> : > : reinstantiate IndexSearcher ,my RAM memory gets appended.is there
>>>> any
>>>> 
>>>> 
>>>> : > IndexSearcher are you explicitly closing both the old IndexSearcher
>>>> as 
>>>> : > well as all of 4 of those old IndexReaders and the MultiReader?
>>>> 
>>>> 
>>>> 
>>>> 
>>>> -Hoss
>>>> 
>>>> 
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>> 
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
> 
> 

-- 
View this message in context: http://www.nabble.com/Java-Heap-Space--Out-Of-Memory-Error-tf4376803.html#a12652816
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message