lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (Updated) (JIRA)" <>
Subject [jira] [Updated] (LUCENE-3588) Try harder to prevent SIGSEGV on cloned MMapIndexInputs
Date Wed, 23 Nov 2011 11:29:39 GMT


Uwe Schindler updated LUCENE-3588:

    Attachment: LUCENE-3588-simpler.patch

Here a much simplier patch than the one yesterday (including Robert's test):
The added complexity by ConcurrentHashMap with WeakReference and ReferenceQueue is nonsense,
as CHM is optimized for many clients getting entries from the map. In our use-case the only
one who gets entries from the map is our close() method. When cloning, we only call put()
so its always synchronized by CHM and no difference to a standard synchronized WhateverMap.
This patch uses the simple apprach: Use a native WeakHashMap where we have a synchronization
on the put()/close() cleanups. This removes all Reference handling and simplifies code a lot.

I think this is ready to commit.
> Try harder to prevent SIGSEGV on cloned MMapIndexInputs
> -------------------------------------------------------
>                 Key: LUCENE-3588
>                 URL:
>             Project: Lucene - Java
>          Issue Type: Improvement
>          Components: core/store
>    Affects Versions: 3.4, 3.5
>            Reporter: Uwe Schindler
>            Assignee: Uwe Schindler
>             Fix For: 3.6, 4.0
>         Attachments: LUCENE-3588-simpler.patch, LUCENE-3588.patch, LUCENE-3588.patch
> We are unmapping mmapped byte buffers which is disallowed by the JDK, because it has
the risk of SIGSEGV when you access the mapped byte buffer after unmapping.
> We currently prevent this for the main IndexInput by setting its buffer to null, so we
NPE if somebody tries to access the underlying buffer. I recently fixed also the stupid curBuf
(LUCENE-3200) by setting to null.
> The big problem are cloned IndexInputs which are generally not closed. Those still contain
references to the unmapped ByteBuffer, which lead to SIGSEGV easily. The patch from Mike in
LUCENE-3439 prevents most of this in Lucene 3.5, but its still not 100% safe (as it uses non-volatiles).
> This patch will fix the remaining issues by also setting the buffers of clones to null
when the original is closed. The trick is to record weak references of all clones created
and close them together with the original. This uses a ConcurrentHashMap<WeakReference<MMapIndexInput>,?>
as store with the logic borrowed from WeakHashMap to cleanup the GCed references (using ReferenceQueue).
> If we respin 3.5, we should maybe also get this in.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:!default.jspa
For more information on JIRA, see:


To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message