From dev-return-122799-apmail-lucene-dev-archive=lucene.apache.org@lucene.apache.org Thu Jan 31 15:55:15 2013 Return-Path: X-Original-To: apmail-lucene-dev-archive@www.apache.org Delivered-To: apmail-lucene-dev-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 81D9BE0D0 for ; Thu, 31 Jan 2013 15:55:15 +0000 (UTC) Received: (qmail 28655 invoked by uid 500); 31 Jan 2013 15:55:14 -0000 Delivered-To: apmail-lucene-dev-archive@lucene.apache.org Received: (qmail 28484 invoked by uid 500); 31 Jan 2013 15:55:13 -0000 Mailing-List: contact dev-help@lucene.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@lucene.apache.org Delivered-To: mailing list dev@lucene.apache.org Received: (qmail 28156 invoked by uid 99); 31 Jan 2013 15:55:12 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 31 Jan 2013 15:55:12 +0000 Date: Thu, 31 Jan 2013 15:55:12 +0000 (UTC) From: "Kristofer Karlsson (JIRA)" To: dev@lucene.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (LUCENE-4740) Weak references cause extreme GC churn MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/LUCENE-4740?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13567734#comment-13567734 ] Kristofer Karlsson commented on LUCENE-4740: -------------------------------------------- I am not sure if this is a problem in practice, but there are also WeakIdentityMaps used in AttributeImpl, AttributeSource, VirtualMethod where the keys of the map are classes, which I don't imagine get allocated or collected a lot at all. I replaced those maps with regular ConcurrentHashMap without any negative impact. > Weak references cause extreme GC churn > -------------------------------------- > > Key: LUCENE-4740 > URL: https://issues.apache.org/jira/browse/LUCENE-4740 > Project: Lucene - Core > Issue Type: Bug > Components: core/index > Affects Versions: 3.6.1 > Environment: Linux debian squeeze 64 bit, Oracle JDK 6, 32 GB RAM, 16 cores > Reporter: Kristofer Karlsson > Priority: Critical > > We are running a set of independent search machines, running our custom software using lucene as a search library. We recently upgraded from lucene 3.0.3 to 3.6.1 and noticed a severe degradation of performance. > After doing some heap dump digging, it turns out the process is stalling because it's spending so much time in GC. We noticed about 212 million WeakReference, originating from WeakIdentityMap, originating from MMapIndexInput. > Our problem completely went away after removing the clones weakhashmap from MMapIndexInput, and as a side-effect, disabling support for explictly unmapping the mmapped data. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscribe@lucene.apache.org For additional commands, e-mail: dev-help@lucene.apache.org