lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sebastin <sebasmt...@gmail.com>
Subject Re: Java Heap Space -Out Of Memory Error
Date Thu, 13 Sep 2007 14:40:30 GMT

java.io.IoException:File Not Found- Segments  is the error message

testn wrote:
> 
> What is the error message? Probably Mike, Erick or Yonik can help you
> better on this since I'm no one in index area.
> 
> Sebastin wrote:
>> 
>> HI testn,
>>              1.I optimize the Large Indexes of size 10 GB using Luke.it
>> optimize all the content into a single CFS file and it generates
>> segments.gen and segments_8 file when i search the item it shows an error
>> that segments file is not there.could you help me in this 
>> 
>> testn wrote:
>>> 
>>> 1. You can close the searcher once you're done. If you want to reopen
>>> the index, you can close and reopen only the updated 3 readers and keep
>>> the 2 old indexreaders and reuse it. It should reduce the time to reopen
>>> it.
>>> 2. Make sure that you optimize it every once in a while
>>> 3. You might consider separating indices in separated storage and use
>>> ParallelReader
>>> 
>>> 
>>> 
>>> Sebastin wrote:
>>>> 
>>>> The problem in my pplication are as follows:
>>>>                  1.I am not able to see the updated records in my index
>>>> store because i instantiate 
>>>> IndexReader and IndexSearcher class once that is in the first
>>>> search.further searches use the same IndexReaders(5 Directories) and
>>>> IndexSearcher with different queries.
>>>> 
>>>>                 2.My search is very very slow First 2 Directories of
>>>> size 10 GB each which are having old index records and no update in
>>>> that remaining 3 Diretories are updated every second.
>>>> 
>>>>                 3.i am Indexing 20 million records per day so the Index
>>>> store gets growing and it makes search very very slower.
>>>>  
>>>>                4.I am using searcherOne class as the global application
>>>> helper class ,with the scope as APPLICATION it consists of one
>>>> IndexReader and IndexSearcher get set method which will hold the
>>>> IndexReader and IndexSearcher object after the First Search.it is used
>>>> for all other searches.
>>>> 
>>>>               5.I am using Lucene 2.2.0 version, in a WEB Application
>>>> which index 15 fields per document and Index 5 Fieds,store 10 Fields.i
>>>> am not using any sort in my query.for a single query upto the maximum
>>>> it fetches 600 records from the index store(5 direcories)    
>>>>                 
>>>> 
>>>> hossman wrote:
>>>>> 
>>>>> 
>>>>> : I set IndexSearcher as the application Object after the first
>>>>> search.
>>>>> 	...
>>>>> : how can i reconstruct the new IndexSearcher for every hour to see
>>>>> the
>>>>> : updated records .
>>>>> 
>>>>> i'm confused ... my understanding based on the comments you made below

>>>>> (in an earlier message) was that you already *were* constructing a new
 
>>>>> IndexSearcher once an hour -- but every time you do that, your memory

>>>>> usage grows, and and that sometimes you got OOM Errors.
>>>>> 
>>>>> if that's not what you said, then i think you need to explain, in
>>>>> detail, 
>>>>> in one message, exactly what your problem is.  And don't assume we 
>>>>> understand anything -- tell us *EVERYTHING* (like, for example, what
>>>>> the 
>>>>> word "crore" means, how "searcherOne" is implemented, and the answer
>>>>> to 
>>>>> the specfic question i asked in my last message: does your
>>>>> application, 
>>>>> contain anywhere in it, any code that will close anything
>>>>> (IndexSearchers 
>>>>> or IndexReaders) ?
>>>>> 
>>>>> 
>>>>> : > : I use StandardAnalyzer.the records daily ranges from 5 crore
to
>>>>> 6 crore.
>>>>> : > for
>>>>> : > : every second i am updating my Index. i instantiate IndexSearcher
>>>>> object
>>>>> : > one
>>>>> : > : time for all the searches. for an hour can i see the updated
>>>>> records in
>>>>> : > the
>>>>> : > : indexstore by reinstantiating IndexSearcher object.but the
>>>>> problem when
>>>>> : > i
>>>>> : > : reinstantiate IndexSearcher ,my RAM memory gets appended.is
>>>>> there any
>>>>> 
>>>>> 
>>>>> : > IndexSearcher are you explicitly closing both the old
>>>>> IndexSearcher as 
>>>>> : > well as all of 4 of those old IndexReaders and the MultiReader?
>>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>> -Hoss
>>>>> 
>>>>> 
>>>>> ---------------------------------------------------------------------
>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
> 
> 

-- 
View this message in context: http://www.nabble.com/Java-Heap-Space--Out-Of-Memory-Error-tf4376803.html#a12655013
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message