lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Muir <rcm...@gmail.com>
Subject Re: Lucene appears to use memory maps after unmapping them
Date Wed, 01 Feb 2012 00:30:27 GMT
the problem is caused by searching indexreaders after you closed them.

in general we can try to add more and more safety, but at the end of the day,
if you close an indexreader while a search is running, you will have problems.

So be careful to only close indexreaders that are no longer in use!

For multithreaded code you might want to investigate IndexReader's
reference counting mechanism or SearcherManager to help you track
this.

On Tue, Jan 31, 2012 at 7:16 PM, Trejkaz <trejkaz@trypticon.org> wrote:
> Hi all.
>
> I've found a rather frustrating issue which I can't seem to get to the
> bottom of.
>
> Our application will crash with an access violation around the time
> when the index is closed, with various indications of what's on the
> stack, but the common things being SegmentTermEnum.next and
> MMapIndexInput.read*.
>
> When it happens, Java doesn't hand us the full stack trace so it's
> impossible to track it back to which of our own pieces of code is
> calling it in this situation.
>
> I tried adding code to MMapIndexInput to check if it has been closed
> at the start of each read method, but that doesn't seem to prevent the
> issue either. Obviously there is some kind of situation where two
> threads are somehow hitting it at the same time so it gets closed
> while some other thread is in the middle of a method call.
>
> I tried changing the code in MMapIndexInput.close to set the array
> element to null before cleaning the mapping, but if it's a
> multi-threading kind of issue this probably won't help anyway as array
> references aren't updated in a thread-safe way (though I guess there
> is scope to experiment with the concurrency classes here.)
>
> I tried turning off the unmapping of the buffers using
> setUseUnmap(true) - this does prevent the crash, but now the index
> files are kept open essentially forever, which prevents other things
> from working.
>
> For now I have just disabled memory mapping entirely by creating a
> SimpleFSDirectory directly instead of going via the factory method. I
> guess this is what the default was originally anyway, so at worst it
> will just mean getting old performance figures back again..?
>
> Annoyingly I can't get it to happen in a self-contained unit test,
> otherwise I would be submitting a proper bug report. :(
>
> TX
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
> For additional commands, e-mail: java-user-help@lucene.apache.org
>



-- 
lucidimagination.com

---------------------------------------------------------------------
To unsubscribe, e-mail: java-user-unsubscribe@lucene.apache.org
For additional commands, e-mail: java-user-help@lucene.apache.org


Mime
View raw message