lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sebastin <sebasmt...@gmail.com>
Subject Re: Java Heap Space -Out Of Memory Error
Date Thu, 13 Sep 2007 15:24:27 GMT

Hi testn,
             i wrote the case wrongly actually the error is 

java.io.ioexception file not found-segments

testn wrote:
> 
> Should the file be "segments_8" and "segments.gen"? Why is it "Segment"?
> The case is different.
> 
> 
> Sebastin wrote:
>> 
>> java.io.IoException:File Not Found- Segments  is the error message
>> 
>> testn wrote:
>>> 
>>> What is the error message? Probably Mike, Erick or Yonik can help you
>>> better on this since I'm no one in index area.
>>> 
>>> Sebastin wrote:
>>>> 
>>>> HI testn,
>>>>              1.I optimize the Large Indexes of size 10 GB using Luke.it
>>>> optimize all the content into a single CFS file and it generates
>>>> segments.gen and segments_8 file when i search the item it shows an
>>>> error that segments file is not there.could you help me in this 
>>>> 
>>>> testn wrote:
>>>>> 
>>>>> 1. You can close the searcher once you're done. If you want to reopen
>>>>> the index, you can close and reopen only the updated 3 readers and
>>>>> keep the 2 old indexreaders and reuse it. It should reduce the time to
>>>>> reopen it.
>>>>> 2. Make sure that you optimize it every once in a while
>>>>> 3. You might consider separating indices in separated storage and use
>>>>> ParallelReader
>>>>> 
>>>>> 
>>>>> 
>>>>> Sebastin wrote:
>>>>>> 
>>>>>> The problem in my pplication are as follows:
>>>>>>                  1.I am not able to see the updated records in my
>>>>>> index store because i instantiate 
>>>>>> IndexReader and IndexSearcher class once that is in the first
>>>>>> search.further searches use the same IndexReaders(5 Directories)
and
>>>>>> IndexSearcher with different queries.
>>>>>> 
>>>>>>                 2.My search is very very slow First 2 Directories
of
>>>>>> size 10 GB each which are having old index records and no update
in
>>>>>> that remaining 3 Diretories are updated every second.
>>>>>> 
>>>>>>                 3.i am Indexing 20 million records per day so the
>>>>>> Index store gets growing and it makes search very very slower.
>>>>>>  
>>>>>>                4.I am using searcherOne class as the global
>>>>>> application helper class ,with the scope as APPLICATION it consists
>>>>>> of one IndexReader and IndexSearcher get set method which will hold
>>>>>> the IndexReader and IndexSearcher object after the First Search.it
is
>>>>>> used for all other searches.
>>>>>> 
>>>>>>               5.I am using Lucene 2.2.0 version, in a WEB Application
>>>>>> which index 15 fields per document and Index 5 Fieds,store 10
>>>>>> Fields.i am not using any sort in my query.for a single query upto
>>>>>> the maximum it fetches 600 records from the index store(5 direcories)
   
>>>>>>                 
>>>>>> 
>>>>>> hossman wrote:
>>>>>>> 
>>>>>>> 
>>>>>>> : I set IndexSearcher as the application Object after the first
>>>>>>> search.
>>>>>>> 	...
>>>>>>> : how can i reconstruct the new IndexSearcher for every hour
to see
>>>>>>> the
>>>>>>> : updated records .
>>>>>>> 
>>>>>>> i'm confused ... my understanding based on the comments you made
>>>>>>> below 
>>>>>>> (in an earlier message) was that you already *were* constructing
a
>>>>>>> new  
>>>>>>> IndexSearcher once an hour -- but every time you do that, your
>>>>>>> memory 
>>>>>>> usage grows, and and that sometimes you got OOM Errors.
>>>>>>> 
>>>>>>> if that's not what you said, then i think you need to explain,
in
>>>>>>> detail, 
>>>>>>> in one message, exactly what your problem is.  And don't assume
we 
>>>>>>> understand anything -- tell us *EVERYTHING* (like, for example,
what
>>>>>>> the 
>>>>>>> word "crore" means, how "searcherOne" is implemented, and the
answer
>>>>>>> to 
>>>>>>> the specfic question i asked in my last message: does your
>>>>>>> application, 
>>>>>>> contain anywhere in it, any code that will close anything
>>>>>>> (IndexSearchers 
>>>>>>> or IndexReaders) ?
>>>>>>> 
>>>>>>> 
>>>>>>> : > : I use StandardAnalyzer.the records daily ranges from
5 crore
>>>>>>> to 6 crore.
>>>>>>> : > for
>>>>>>> : > : every second i am updating my Index. i instantiate
>>>>>>> IndexSearcher object
>>>>>>> : > one
>>>>>>> : > : time for all the searches. for an hour can i see the
updated
>>>>>>> records in
>>>>>>> : > the
>>>>>>> : > : indexstore by reinstantiating IndexSearcher object.but
the
>>>>>>> problem when
>>>>>>> : > i
>>>>>>> : > : reinstantiate IndexSearcher ,my RAM memory gets appended.is
>>>>>>> there any
>>>>>>> 
>>>>>>> 
>>>>>>> : > IndexSearcher are you explicitly closing both the old
>>>>>>> IndexSearcher as 
>>>>>>> : > well as all of 4 of those old IndexReaders and the MultiReader?
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> -Hoss
>>>>>>> 
>>>>>>> 
>>>>>>> ---------------------------------------------------------------------
>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
> 
> 

-- 
View this message in context: http://www.nabble.com/Java-Heap-Space--Out-Of-Memory-Error-tf4376803.html#a12657227
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message