lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From testn <te...@doramail.com>
Subject Re: Java Heap Space -Out Of Memory Error
Date Fri, 14 Sep 2007 17:13:46 GMT

So did you see any improvement in performance?

Sebastin wrote:
> 
> It works finally .i use Lucene 2.2  in my application.thanks testn and
> Mike
> 
> Michael McCandless-2 wrote:
>> 
>> 
>> It sounds like there may be a Lucene version mismatch?  When Luke was
>> used
>> it was likely based on Lucene 2.2, but it sounds like an older version of
>> Lucene is now being used to open the index?
>> 
>> Mike
>> 
>> "testn" <test1@doramail.com> wrote:
>>> 
>>> Should the file be "segments_8" and "segments.gen"? Why is it "Segment"?
>>> The
>>> case is different.
>>> 
>>> 
>>> Sebastin wrote:
>>> > 
>>> > java.io.IoException:File Not Found- Segments  is the error message
>>> > 
>>> > testn wrote:
>>> >> 
>>> >> What is the error message? Probably Mike, Erick or Yonik can help you
>>> >> better on this since I'm no one in index area.
>>> >> 
>>> >> Sebastin wrote:
>>> >>> 
>>> >>> HI testn,
>>> >>>              1.I optimize the Large Indexes of size 10 GB using
>>> Luke.it
>>> >>> optimize all the content into a single CFS file and it generates
>>> >>> segments.gen and segments_8 file when i search the item it shows
an
>>> >>> error that segments file is not there.could you help me in this

>>> >>> 
>>> >>> testn wrote:
>>> >>>> 
>>> >>>> 1. You can close the searcher once you're done. If you want
to
>>> reopen
>>> >>>> the index, you can close and reopen only the updated 3 readers
and
>>> keep
>>> >>>> the 2 old indexreaders and reuse it. It should reduce the time
to
>>> >>>> reopen it.
>>> >>>> 2. Make sure that you optimize it every once in a while
>>> >>>> 3. You might consider separating indices in separated storage
and
>>> use
>>> >>>> ParallelReader
>>> >>>> 
>>> >>>> 
>>> >>>> 
>>> >>>> Sebastin wrote:
>>> >>>>> 
>>> >>>>> The problem in my pplication are as follows:
>>> >>>>>                  1.I am not able to see the updated records
in my
>>> >>>>> index store because i instantiate 
>>> >>>>> IndexReader and IndexSearcher class once that is in the
first
>>> >>>>> search.further searches use the same IndexReaders(5 Directories)
>>> and
>>> >>>>> IndexSearcher with different queries.
>>> >>>>> 
>>> >>>>>                 2.My search is very very slow First 2 Directories
>>> of
>>> >>>>> size 10 GB each which are having old index records and no
update
>>> in
>>> >>>>> that remaining 3 Diretories are updated every second.
>>> >>>>> 
>>> >>>>>                 3.i am Indexing 20 million records per day
so the
>>> >>>>> Index store gets growing and it makes search very very slower.
>>> >>>>>  
>>> >>>>>                4.I am using searcherOne class as the global
>>> >>>>> application helper class ,with the scope as APPLICATION
it
>>> consists of
>>> >>>>> one IndexReader and IndexSearcher get set method which will
hold
>>> the
>>> >>>>> IndexReader and IndexSearcher object after the First Search.it
is
>>> used
>>> >>>>> for all other searches.
>>> >>>>> 
>>> >>>>>               5.I am using Lucene 2.2.0 version, in a WEB
>>> Application
>>> >>>>> which index 15 fields per document and Index 5 Fieds,store
10
>>> Fields.i
>>> >>>>> am not using any sort in my query.for a single query upto
the
>>> maximum
>>> >>>>> it fetches 600 records from the index store(5 direcories)
   
>>> >>>>>                 
>>> >>>>> 
>>> >>>>> hossman wrote:
>>> >>>>>> 
>>> >>>>>> 
>>> >>>>>> : I set IndexSearcher as the application Object after
the first
>>> >>>>>> search.
>>> >>>>>> 	...
>>> >>>>>> : how can i reconstruct the new IndexSearcher for every
hour to
>>> see
>>> >>>>>> the
>>> >>>>>> : updated records .
>>> >>>>>> 
>>> >>>>>> i'm confused ... my understanding based on the comments
you made
>>> >>>>>> below 
>>> >>>>>> (in an earlier message) was that you already *were*
constructing
>>> a
>>> >>>>>> new  
>>> >>>>>> IndexSearcher once an hour -- but every time you do
that, your
>>> memory 
>>> >>>>>> usage grows, and and that sometimes you got OOM Errors.
>>> >>>>>> 
>>> >>>>>> if that's not what you said, then i think you need to
explain, in
>>> >>>>>> detail, 
>>> >>>>>> in one message, exactly what your problem is.  And don't
assume
>>> we 
>>> >>>>>> understand anything -- tell us *EVERYTHING* (like, for
example,
>>> what
>>> >>>>>> the 
>>> >>>>>> word "crore" means, how "searcherOne" is implemented,
and the
>>> answer
>>> >>>>>> to 
>>> >>>>>> the specfic question i asked in my last message: does
your
>>> >>>>>> application, 
>>> >>>>>> contain anywhere in it, any code that will close anything
>>> >>>>>> (IndexSearchers 
>>> >>>>>> or IndexReaders) ?
>>> >>>>>> 
>>> >>>>>> 
>>> >>>>>> : > : I use StandardAnalyzer.the records daily ranges
from 5
>>> crore to
>>> >>>>>> 6 crore.
>>> >>>>>> : > for
>>> >>>>>> : > : every second i am updating my Index. i instantiate
>>> >>>>>> IndexSearcher object
>>> >>>>>> : > one
>>> >>>>>> : > : time for all the searches. for an hour can
i see the
>>> updated
>>> >>>>>> records in
>>> >>>>>> : > the
>>> >>>>>> : > : indexstore by reinstantiating IndexSearcher
object.but the
>>> >>>>>> problem when
>>> >>>>>> : > i
>>> >>>>>> : > : reinstantiate IndexSearcher ,my RAM memory
gets appended.is
>>> >>>>>> there any
>>> >>>>>> 
>>> >>>>>> 
>>> >>>>>> : > IndexSearcher are you explicitly closing both
the old
>>> >>>>>> IndexSearcher as 
>>> >>>>>> : > well as all of 4 of those old IndexReaders and
the
>>> MultiReader?
>>> >>>>>> 
>>> >>>>>> 
>>> >>>>>> 
>>> >>>>>> 
>>> >>>>>> -Hoss
>>> >>>>>> 
>>> >>>>>> 
>>> >>>>>>
>>> ---------------------------------------------------------------------
>>> >>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> >>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>> >>>>>> 
>>> >>>>>> 
>>> >>>>>> 
>>> >>>>> 
>>> >>>>> 
>>> >>>> 
>>> >>>> 
>>> >>> 
>>> >>> 
>>> >> 
>>> >> 
>>> > 
>>> > 
>>> 
>>> -- 
>>> View this message in context:
>>> http://www.nabble.com/Java-Heap-Space--Out-Of-Memory-Error-tf4376803.html#a12655880
>>> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
>>> 
>>> 
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>>> For additional commands, e-mail: java-user-help@lucene.apache.org
>>> 
>> 
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
>> For additional commands, e-mail: java-user-help@lucene.apache.org
>> 
>> 
>> 
> 
> 

-- 
View this message in context: http://www.nabble.com/Java-Heap-Space--Out-Of-Memory-Error-tf4376803.html#a12679970
Sent from the Lucene - Java Users mailing list archive at Nabble.com.


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message