lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Michael McCandless" <luc...@mikemccandless.com>
Subject Re: Java Heap Space -Out Of Memory Error
Date Thu, 13 Sep 2007 14:53:16 GMT

It sounds like there may be a Lucene version mismatch?  When Luke was used
it was likely based on Lucene 2.2, but it sounds like an older version of
Lucene is now being used to open the index?

Mike

"testn" <test1@doramail.com> wrote:
> 
> Should the file be "segments_8" and "segments.gen"? Why is it "Segment"?
> The
> case is different.
> 
> 
> Sebastin wrote:
> > 
> > java.io.IoException:File Not Found- Segments  is the error message
> > 
> > testn wrote:
> >> 
> >> What is the error message? Probably Mike, Erick or Yonik can help you
> >> better on this since I'm no one in index area.
> >> 
> >> Sebastin wrote:
> >>> 
> >>> HI testn,
> >>>              1.I optimize the Large Indexes of size 10 GB using Luke.it
> >>> optimize all the content into a single CFS file and it generates
> >>> segments.gen and segments_8 file when i search the item it shows an
> >>> error that segments file is not there.could you help me in this 
> >>> 
> >>> testn wrote:
> >>>> 
> >>>> 1. You can close the searcher once you're done. If you want to reopen
> >>>> the index, you can close and reopen only the updated 3 readers and keep
> >>>> the 2 old indexreaders and reuse it. It should reduce the time to
> >>>> reopen it.
> >>>> 2. Make sure that you optimize it every once in a while
> >>>> 3. You might consider separating indices in separated storage and use
> >>>> ParallelReader
> >>>> 
> >>>> 
> >>>> 
> >>>> Sebastin wrote:
> >>>>> 
> >>>>> The problem in my pplication are as follows:
> >>>>>                  1.I am not able to see the updated records in my
> >>>>> index store because i instantiate 
> >>>>> IndexReader and IndexSearcher class once that is in the first
> >>>>> search.further searches use the same IndexReaders(5 Directories)
and
> >>>>> IndexSearcher with different queries.
> >>>>> 
> >>>>>                 2.My search is very very slow First 2 Directories
of
> >>>>> size 10 GB each which are having old index records and no update
in
> >>>>> that remaining 3 Diretories are updated every second.
> >>>>> 
> >>>>>                 3.i am Indexing 20 million records per day so the
> >>>>> Index store gets growing and it makes search very very slower.
> >>>>>  
> >>>>>                4.I am using searcherOne class as the global
> >>>>> application helper class ,with the scope as APPLICATION it consists
of
> >>>>> one IndexReader and IndexSearcher get set method which will hold
the
> >>>>> IndexReader and IndexSearcher object after the First Search.it is
used
> >>>>> for all other searches.
> >>>>> 
> >>>>>               5.I am using Lucene 2.2.0 version, in a WEB Application
> >>>>> which index 15 fields per document and Index 5 Fieds,store 10 Fields.i
> >>>>> am not using any sort in my query.for a single query upto the maximum
> >>>>> it fetches 600 records from the index store(5 direcories)    
> >>>>>                 
> >>>>> 
> >>>>> hossman wrote:
> >>>>>> 
> >>>>>> 
> >>>>>> : I set IndexSearcher as the application Object after the first
> >>>>>> search.
> >>>>>> 	...
> >>>>>> : how can i reconstruct the new IndexSearcher for every hour
to see
> >>>>>> the
> >>>>>> : updated records .
> >>>>>> 
> >>>>>> i'm confused ... my understanding based on the comments you
made
> >>>>>> below 
> >>>>>> (in an earlier message) was that you already *were* constructing
a
> >>>>>> new  
> >>>>>> IndexSearcher once an hour -- but every time you do that, your
memory 
> >>>>>> usage grows, and and that sometimes you got OOM Errors.
> >>>>>> 
> >>>>>> if that's not what you said, then i think you need to explain,
in
> >>>>>> detail, 
> >>>>>> in one message, exactly what your problem is.  And don't assume
we 
> >>>>>> understand anything -- tell us *EVERYTHING* (like, for example,
what
> >>>>>> the 
> >>>>>> word "crore" means, how "searcherOne" is implemented, and the
answer
> >>>>>> to 
> >>>>>> the specfic question i asked in my last message: does your
> >>>>>> application, 
> >>>>>> contain anywhere in it, any code that will close anything
> >>>>>> (IndexSearchers 
> >>>>>> or IndexReaders) ?
> >>>>>> 
> >>>>>> 
> >>>>>> : > : I use StandardAnalyzer.the records daily ranges from
5 crore to
> >>>>>> 6 crore.
> >>>>>> : > for
> >>>>>> : > : every second i am updating my Index. i instantiate
> >>>>>> IndexSearcher object
> >>>>>> : > one
> >>>>>> : > : time for all the searches. for an hour can i see the
updated
> >>>>>> records in
> >>>>>> : > the
> >>>>>> : > : indexstore by reinstantiating IndexSearcher object.but
the
> >>>>>> problem when
> >>>>>> : > i
> >>>>>> : > : reinstantiate IndexSearcher ,my RAM memory gets appended.is
> >>>>>> there any
> >>>>>> 
> >>>>>> 
> >>>>>> : > IndexSearcher are you explicitly closing both the old
> >>>>>> IndexSearcher as 
> >>>>>> : > well as all of 4 of those old IndexReaders and the MultiReader?
> >>>>>> 
> >>>>>> 
> >>>>>> 
> >>>>>> 
> >>>>>> -Hoss
> >>>>>> 
> >>>>>> 
> >>>>>> ---------------------------------------------------------------------
> >>>>>> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> >>>>>> For additional commands, e-mail: java-user-help@lucene.apache.org
> >>>>>> 
> >>>>>> 
> >>>>>> 
> >>>>> 
> >>>>> 
> >>>> 
> >>>> 
> >>> 
> >>> 
> >> 
> >> 
> > 
> > 
> 
> -- 
> View this message in context:
> http://www.nabble.com/Java-Heap-Space--Out-Of-Memory-Error-tf4376803.html#a12655880
> Sent from the Lucene - Java Users mailing list archive at Nabble.com.
> 
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
> 

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message