lucene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Uwe Schindler (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (LUCENE-4740) Weak references cause extreme GC churn
Date Fri, 01 Feb 2013 10:25:12 GMT

    [ https://issues.apache.org/jira/browse/LUCENE-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13568637#comment-13568637
] 

Uwe Schindler commented on LUCENE-4740:
---------------------------------------

bq. I would prefer an AtomicBoolean since it uses a volatile field. As far as I know, you
can't make contents of arrays volatile.

This kills performance. MMapIndexInput would be slower than SimpleFSIndexInput! This is why
the array is used as a fake "reference" to a boolean.

The current approach of unmapping the byte buffers and protecting for sigsegv by nulling them
is not 100% safe. The JVM may still crash if another thread does not yet see the nulled buffer.
But in *most* cases the use will get a AlreadyClosedException and can fix his code before
he goes into production and his JVM crashes suddenly.
                
> Weak references cause extreme GC churn
> --------------------------------------
>
>                 Key: LUCENE-4740
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4740
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: core/store
>    Affects Versions: 3.6.1
>         Environment: Linux debian squeeze 64 bit, Oracle JDK 6, 32 GB RAM, 16 cores
>            Reporter: Kristofer Karlsson
>            Priority: Critical
>         Attachments: LUCENE-4740.patch
>
>
> We are running a set of independent search machines, running our custom software using
lucene as a search library. We recently upgraded from lucene 3.0.3 to 3.6.1 and noticed a
severe degradation of performance.
> After doing some heap dump digging, it turns out the process is stalling because it's
spending so much time in GC. We noticed about 212 million WeakReference, originating from
WeakIdentityMap, originating from MMapIndexInput.
> Our problem completely went away after removing the clones weakhashmap from MMapIndexInput,
and as a side-effect, disabling support for explictly unmapping the mmapped data.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org


Mime
View raw message