Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Message-ID: <210683158.398271269237147160.JavaMail.jira@brutus.apache.org>
Date: Mon, 22 Mar 2010 05:52:27 +0000 (UTC)
From: "Chris Goffinet (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Subject: [jira] Commented: (CASSANDRA-896) GC issues with
 LinkedBlockingQueue, very large heap sizes
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/CASSANDRA-896?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12848031#action_12848031 ] 

Chris Goffinet commented on CASSANDRA-896:
------------------------------------------

ArrayBlockingQueue helps keep memory under control. I set the capacity to 4096 (stagingQueueSize). Test is still running but within first 15 minutes its stable. I will measure different sizes, as this is going to limit writes (blocking queue). I think we will have to make this a tunable so users can trade off memory for higher write throughput.


> GC issues with LinkedBlockingQueue, very large heap sizes
> ---------------------------------------------------------
>
>                 Key: CASSANDRA-896
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-896
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.6
>            Reporter: Chris Goffinet
>            Assignee: Chris Goffinet
>             Fix For: 0.6
>
>         Attachments: Screen shot 2010-03-14 at 10.19.09 PM.png, Screen shot 2010-03-14 at 10.20.31 PM.png
>
>
> We were doing a large import over thrift today (250M rows) with 50 columns each. We noticed when this process started, that the heap usage slowly began to rise. Eventually hitting 10GB. After investigating heap dumps we noticed this is linked to GC problems in the JVM for LinkedBlockingQueue. Sending a hint through jconsole to perform GC didn't help either.
> See this article:
> http://tech.puredanger.com/2009/02/11/linkedblockingqueue-garbagecollection/
> JDK  1.6.18 doesn't have this fix yet either. We could possibly add the change from:
> http://svn.apache.org/repos/asf/harmony/standard/classlib/trunk/modules/concurrent/src/main/java/java/util/concurrent/LinkedBlockingQueue.java

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.