Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Wed, 30 Oct 2013 08:47:25 +0000 (UTC)
From: "Benedict (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12676575.1383122452288.4010.1383122845385@arcas>
In-Reply-To: <JIRA.12676575.1383122452288@arcas>
References: <JIRA.12676575.1383122452288@arcas>
Subject: [jira] [Updated] (CASSANDRA-6271) Replace SnapTree in
 AtomicSortedColumns
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


     [ https://issues.apache.org/jira/browse/CASSANDRA-6271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Benedict updated CASSANDRA-6271:
--------------------------------

    Description: 
On the write path a huge percentage of time is spent in GC (>50% in my tests, if accounting for slow down due to parallel marking). SnapTrees are both GC unfriendly due to their structure and also very expensive to keep around - each column name in AtomicSortedColumns uses > 100 bytes on average.

I suggest using a sorted array; changes are supplied at-once, as opposed to one at a time, and if < 10% of the values in the array change (and data equal to < 10% of the size of the key array) we simply overlay a new array of changes only over the top. Otherwise we rewrite the array. This method should ensure much less GC overhead, and also save approximately 80% of the current memory overhead.

TreeMap is similarly difficult object for the GC, and a related task might be to remove it where not strictly necessary, even though we don't keep them hanging around for long. TreeMapBackedSortedColumns, for instance, seems to be used in a lot of places where we could simply sort the columns.

  was:
On the write path a huge percentage of time is spent in GC (>50% in my tests, if accounting for slow down due to parallel marking). SnapTrees are both GC unfriendly due to their structure and also very expensive to keep around - each column name in AtomicSortedColumns uses > 100 bytes on average.

I suggest using a sorted array; changes are supplied at-once, as opposed to one at a time, and if < 10% of the values in the array change we simply overlay a new array of changes only over the top. Otherwise we rewrite the array. This method should ensure much less GC overhead, and also save approximately 80% of the current memory overhead.

TreeMap is similarly difficult object for the GC, and a related task might be to remove it where not strictly necessary, even though we don't keep them hanging around for long. TreeMapBackedSortedColumns, for instance, seems to be used in a lot of places where we could simply sort the columns.


> Replace SnapTree in AtomicSortedColumns
> ---------------------------------------
>
>                 Key: CASSANDRA-6271
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-6271
>             Project: Cassandra
>          Issue Type: Improvement
>            Reporter: Benedict
>            Assignee: Benedict
>
> On the write path a huge percentage of time is spent in GC (>50% in my tests, if accounting for slow down due to parallel marking). SnapTrees are both GC unfriendly due to their structure and also very expensive to keep around - each column name in AtomicSortedColumns uses > 100 bytes on average.
> I suggest using a sorted array; changes are supplied at-once, as opposed to one at a time, and if < 10% of the values in the array change (and data equal to < 10% of the size of the key array) we simply overlay a new array of changes only over the top. Otherwise we rewrite the array. This method should ensure much less GC overhead, and also save approximately 80% of the current memory overhead.
> TreeMap is similarly difficult object for the GC, and a related task might be to remove it where not strictly necessary, even though we don't keep them hanging around for long. TreeMapBackedSortedColumns, for instance, seems to be used in a lot of places where we could simply sort the columns.


--
This message was sent by Atlassian JIRA
(v6.1#6144)