Return-Path: X-Original-To: apmail-cassandra-commits-archive@www.apache.org Delivered-To: apmail-cassandra-commits-archive@www.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 0072010700 for ; Wed, 1 Jan 2014 19:07:50 +0000 (UTC) Received: (qmail 89913 invoked by uid 500); 1 Jan 2014 19:07:50 -0000 Delivered-To: apmail-cassandra-commits-archive@cassandra.apache.org Received: (qmail 89891 invoked by uid 500); 1 Jan 2014 19:07:50 -0000 Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@cassandra.apache.org Delivered-To: mailing list commits@cassandra.apache.org Received: (qmail 89881 invoked by uid 99); 1 Jan 2014 19:07:50 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 01 Jan 2014 19:07:50 +0000 Date: Wed, 1 Jan 2014 19:07:50 +0000 (UTC) From: "Benedict (JIRA)" To: commits@cassandra.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (CASSANDRA-6271) Replace SnapTree in AtomicSortedColumns MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/CASSANDRA-6271?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13859925#comment-13859925 ] Benedict commented on CASSANDRA-6271: ------------------------------------- bq. Do we actually win anything by making Cursors reset-able? Looks to me like the main consumer is BTree.slice which always creates a new Cursor. Well, it opens up some potential optimisations. We might want to optimise the Slice iterator to use one cursor, for instance. I didn't want to get too bogged down in those sorts of optimisations this pass, but it keeps scope open. > Replace SnapTree in AtomicSortedColumns > --------------------------------------- > > Key: CASSANDRA-6271 > URL: https://issues.apache.org/jira/browse/CASSANDRA-6271 > Project: Cassandra > Issue Type: Improvement > Reporter: Benedict > Assignee: Benedict > Labels: performance > Attachments: oprate.svg > > > On the write path a huge percentage of time is spent in GC (>50% in my tests, if accounting for slow down due to parallel marking). SnapTrees are both GC unfriendly due to their structure and also very expensive to keep around - each column name in AtomicSortedColumns uses > 100 bytes on average (excluding the actual ByteBuffer). > I suggest using a sorted array; changes are supplied at-once, as opposed to one at a time, and if < 10% of the keys in the array change (and data equal to < 10% of the size of the key array) we simply overlay a new array of changes only over the top. Otherwise we rewrite the array. This method should ensure much less GC overhead, and also save approximately 80% of the current memory overhead. > TreeMap is similarly difficult object for the GC, and a related task might be to remove it where not strictly necessary, even though we don't keep them hanging around for long. TreeMapBackedSortedColumns, for instance, seems to be used in a lot of places where we could simply sort the columns. -- This message was sent by Atlassian JIRA (v6.1.5#6160)