lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From testn <te...@doramail.com>
Subject Re: Java Heap Space -Out Of Memory Error
Date Thu, 13 Sep 2007 15:27:43 GMT

As Mike mentioned, what is the version of Lucene you are using? Plus can you
also post the stacktrace?


Sebastin wrote:
> 
> Hi testn,
>              i wrote the case wrongly actually the error is 
> 
> java.io.ioexception file not found-segments
> 
> testn wrote:
>> 
>> Should the file be "segments_8" and "segments.gen"? Why is it "Segment"?
>> The case is different.
>> 
>> 
>> Sebastin wrote:
>>> 
>>> java.io.IoException:File Not Found- Segments  is the error message
>>> 
>>> testn wrote:
>>>> 
>>>> What is the error message? Probably Mike, Erick or Yonik can help you
>>>> better on this since I'm no one in index area.
>>>> 
>>>> Sebastin wrote:
>>>>> 
>>>>> HI testn,
>>>>>              1.I optimize the Large Indexes of size 10 GB using
>>>>> Luke.it optimize all the content into a single CFS file and it
>>>>> generates segments.gen and segments_8 file when i search the item it
>>>>> shows an error that segments file is not there.could you help me in
>>>>> this 
>>>>> 
>>>>> testn wrote:
>>>>>> 
>>>>>> 1. You can close the searcher once you're done. If you want to reopen
>>>>>> the index, you can close and reopen only the updated 3 readers and
>>>>>> keep the 2 old indexreaders and reuse it. It should reduce the time
>>>>>> to reopen it.
>>>>>> 2. Make sure that you optimize it every once in a while
>>>>>> 3. You might consider separating indices in separated storage and
use
>>>>>> ParallelReader
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> Sebastin wrote:
>>>>>>> 
>>>>>>> The problem in my pplication are as follows:
>>>>>>>                  1.I am not able to see the updated records in
my
>>>>>>> index store because i instantiate 
>>>>>>> IndexReader and IndexSearcher class once that is in the first
>>>>>>> search.further searches use the same IndexReaders(5 Directories)
and
>>>>>>> IndexSearcher with different queries.
>>>>>>> 
>>>>>>>                 2.My search is very very slow First 2 Directories
of
>>>>>>> size 10 GB each which are having old index records and no update
in
>>>>>>> that remaining 3 Diretories are updated every second.
>>>>>>> 
>>>>>>>                 3.i am Indexing 20 million records per day so
the
>>>>>>> Index store gets growing and it makes search very very slower.
>>>>>>>  
>>>>>>>                4.I am using searcherOne class as the global
>>>>>>> application helper class ,with the scope as APPLICATION it consists
>>>>>>> of one IndexReader and IndexSearcher get set method which will
hold
>>>>>>> the IndexReader and IndexSearcher object after the First Search.it
>>>>>>> is used for all other searches.
>>>>>>> 
>>>>>>>               5.I am using Lucene 2.2.0 version, in a WEB
>>>>>>> Application which index 15 fields per document and Index 5
>>>>>>> Fieds,store 10 Fields.i am not using any sort in my query.for
a
>>>>>>> single query upto the maximum it fetches 600 records from the
index
>>>>>>> store(5 direcories)    
>>>>>>>                 
>>>>>>> 
>>>>>>> hossman wrote:
>>>>>>>> 
>>>>>>>> 
>>>>>>>> : I set IndexSearcher as the application Object after the
first
>>>>>>>> search.
>>>>>>>> 	...
>>>>>>>> : how can i reconstruct the new IndexSearcher for every hour
to see
>>>>>>>> the
>>>>>>>> : updated records .
>>>>>>>> 
>>>>>>>> i'm confused ... my understanding based on the comments you
made
>>>>>>>> below 
>>>>>>>> (in an earlier message) was that you already *were* constructing
a
>>>>>>>> new  
>>>>>>>> IndexSearcher once an hour -- but every time you do that,
your
>>>>>>>> memory 
>>>>>>>> usage grows, and and that sometimes you got OOM Errors.
>>>>>>>> 
>>>>>>>> if that's not what you said, then i think you need to explain,
in
>>>>>>>> detail, 
>>>>>>>> in one message, exactly what your problem is.  And don't
assume we 
>>>>>>>> understand anything -- tell us *EVERYTHING* (like, for example,
>>>>>>>> what the 
>>>>>>>> word "crore" means, how "searcherOne" is implemented, and
the
>>>>>>>> answer to 
>>>>>>>> the specfic question i asked in my last message: does your
>>>>>>>> application, 
>>>>>>>> contain anywhere in it, any code that will close anything
>>>>>>>> (IndexSearchers 
>>>>>>>> or IndexReaders) ?
>>>>>>>> 
>>>>>>>> 
>>>>>>>> : > : I use StandardAnalyzer.the records daily ranges
from 5 crore
>>>>>>>> to 6 crore.
>>>>>>>> : > for
>>>>>>>> : > : every second i am updating my Index. i instantiate
>>>>>>>> IndexSearcher object
>>>>>>>> : > one
>>>>>>>> : > : time for all the searches. for an hour can i see
the updated
>>>>>>>> records in
>>>>>>>> : > the
>>>>>>>> : > : indexstore by reinstantiating IndexSearcher object.but
the
>>>>>>>> problem when
>>>>>>>> : > i
>>>>>>>> : > : reinstantiate IndexSearcher ,my RAM memory gets
appended.is
>>>>>>>> there any
>>>>>>>> 
>>>>>>>> 
>>>>>>>> : > IndexSearcher are you explicitly closing both the
old
>>>>>>>> IndexSearcher as 
>>>>>>>> : > well as all of 4 of those old IndexReaders and the
MultiReader?
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>>> -Hoss
>>>>>>>> 
>>>>>>>> 
>>>>>>>> ---------------------------------------------------------------------
>>>>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>>>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>>>>>>> 
>>>>>>>> 
>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>> 
>>>>>> 
>>>>> 
>>>>> 
>>>> 
>>>> 
>>> 
>>> 
>> 
>> 
> 
> 

-- 
View this message in context: http://www.nabble.com/Java-Heap-Space--Out-Of-Memory-Error-tf4376803.html#a12657298
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message