lucene-java-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Trejkaz <>
Subject Lucene appears to use memory maps after unmapping them
Date Wed, 01 Feb 2012 00:16:25 GMT
Hi all.

I've found a rather frustrating issue which I can't seem to get to the
bottom of.

Our application will crash with an access violation around the time
when the index is closed, with various indications of what's on the
stack, but the common things being and*.

When it happens, Java doesn't hand us the full stack trace so it's
impossible to track it back to which of our own pieces of code is
calling it in this situation.

I tried adding code to MMapIndexInput to check if it has been closed
at the start of each read method, but that doesn't seem to prevent the
issue either. Obviously there is some kind of situation where two
threads are somehow hitting it at the same time so it gets closed
while some other thread is in the middle of a method call.

I tried changing the code in MMapIndexInput.close to set the array
element to null before cleaning the mapping, but if it's a
multi-threading kind of issue this probably won't help anyway as array
references aren't updated in a thread-safe way (though I guess there
is scope to experiment with the concurrency classes here.)

I tried turning off the unmapping of the buffers using
setUseUnmap(true) - this does prevent the crash, but now the index
files are kept open essentially forever, which prevents other things
from working.

For now I have just disabled memory mapping entirely by creating a
SimpleFSDirectory directly instead of going via the factory method. I
guess this is what the default was originally anyway, so at worst it
will just mean getting old performance figures back again..?

Annoyingly I can't get it to happen in a self-contained unit test,
otherwise I would be submitting a proper bug report. :(


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message