lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler" <...@thetaphi.de>
Subject RE: Lucene appears to use memory maps after unmapping them
Date Wed, 01 Feb 2012 11:44:53 GMT
Hi one addition:
In the coming Lucene 3.6 there are more safety checks in MMapDirectory so the SIGSEGV is more
unlikely (it tracks cloned index input in a thread safe list on close). But this only *helps*
to find the issue, but does not guarantee that your JVM crashes, sorry.

As Robert and Mike noted, check that you app don’t close IndexReaders that are still in
use by other threads. SearcherManager is a good idea. Alternatively use refcounting:
- Open IndexReader (this increments its ref count automatically to 1)
- every thread that uses the reader does IR.incRef() and when done does IR.decRef(), ideally
using try...finally
- when the main thread wants to recycle the reader it can simply do this by also calling IR.decRef()

This way only when all incRefs have corresponding decRef (the open itself is also an incRef
internally) the reader is really closed (it should at the end have IR.getRefCount()==0).

Uwe

-----
Uwe Schindler
H.-H.-Meier-Allee 63, D-28213 Bremen
http://www.thetaphi.de
eMail: uwe@thetaphi.de


> -----Original Message-----
> From: Trejkaz [mailto:trejkaz@trypticon.org]
> Sent: Wednesday, February 01, 2012 2:33 AM
> To: java-user@lucene.apache.org
> Subject: Re: Lucene appears to use memory maps after unmapping them
> 
> On Wed, Feb 1, 2012 at 11:30 AM, Robert Muir <rcmuir@gmail.com> wrote:
> > the problem is caused by searching indexreaders after you closed them.
> >
> > in general we can try to add more and more safety, but at the end of
> > the day, if you close an indexreader while a search is running, you will have
> problems.
> >
> > So be careful to only close indexreaders that are no longer in use!
> >
> > For multithreaded code you might want to investigate IndexReader's
> > reference counting mechanism or SearcherManager to help you track
> > this.
> 
> So rather than flagging closed = true when close() is called and hitting someone
> who tries to read data, essentially try and do the
> reverse- if a search is still running and close() is called, throw an exception to
> them?
> 
> Some of our underlying code isn't only doing searches... so it still might not
> catch all usages. But it should catch the more easily found ones I hope.
> 
> I guess I would need to create a stack trace every time someone starts a new
> search for this to work, so that there is some point of blame when the
> exception fires. Otherwise I would be back at step 1, having no idea who did the
> wrong thing.
> 
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org


---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message