Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@lucene.apache.org
Date: Thu, 31 Jan 2013 18:11:13 +0000 (UTC)
From: "Kristofer Karlsson (JIRA)" <jira@apache.org>
To: dev@lucene.apache.org
Message-ID: <JIRA.12630159.1359647384375.223653.1359655873156@arcas>
In-Reply-To: <JIRA.12630159.1359647384375@arcas>
References: <JIRA.12630159.1359647384375@arcas>
Subject: [jira] [Commented] (LUCENE-4740) Weak references cause extreme GC
 churn
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/LUCENE-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13567887#comment-13567887 ] 

Kristofer Karlsson commented on LUCENE-4740:
--------------------------------------------

Yes, that change is probably not a good general solution, but it worked well for our usecase. It might be nice to have support for unloadable classes optional.

I am using debian squeeze, 64 bit, Oracle JDK 6 with a 16 GB heap and four indexes, totalling 9 GB
{noformat}
java version "1.6.0_26"
Java(TM) SE Runtime Environment (build 1.6.0_26-b03)
Java HotSpot(TM) 64-Bit Server VM (build 20.1-b02, mixed mode)
{noformat}

The 212M WeakReferences on the heap indicates that the referencequeues weren't functioning correctly.

In any case, if the useUnmap is false, then it seems unnecessary to even add references to the clones to the map.
                
> Weak references cause extreme GC churn
> --------------------------------------
>
>                 Key: LUCENE-4740
>                 URL: https://issues.apache.org/jira/browse/LUCENE-4740
>             Project: Lucene - Core
>          Issue Type: Bug
>          Components: core/store
>    Affects Versions: 3.6.1
>         Environment: Linux debian squeeze 64 bit, Oracle JDK 6, 32 GB RAM, 16 cores
>            Reporter: Kristofer Karlsson
>            Priority: Critical
>
> We are running a set of independent search machines, running our custom software using lucene as a search library. We recently upgraded from lucene 3.0.3 to 3.6.1 and noticed a severe degradation of performance.
> After doing some heap dump digging, it turns out the process is stalling because it's spending so much time in GC. We noticed about 212 million WeakReference, originating from WeakIdentityMap, originating from MMapIndexInput.
> Our problem completely went away after removing the clones weakhashmap from MMapIndexInput, and as a side-effect, disabling support for explictly unmapping the mmapped data.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

---------------------------------------------------------------------
To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org
For additional commands, e-mail: dev-help@lucene.apache.org