Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@cassandra.apache.org
Date: Tue, 20 Sep 2016 09:25:20 +0000 (UTC)
From: "Benedict (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.13006017.1474313479000.614516.1474363520524@Atlassian.JIRA>
In-Reply-To: <JIRA.13006017.1474313479000@Atlassian.JIRA>
References: <JIRA.13006017.1474313479000@Atlassian.JIRA> <JIRA.13006017.1474313479286@arcas>
Subject: [jira] [Comment Edited] (CASSANDRA-12668) Memtable Contention in
 2.1
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Tue, 20 Sep 2016 09:25:22 -0000


    [ https://issues.apache.org/jira/browse/CASSANDRA-12668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15506130#comment-15506130 ] 

Benedict edited comment on CASSANDRA-12668 at 9/20/16 9:24 AM:
---------------------------------------------------------------

So, just to crystalise my thoughts on this.  Given this presents with contention, there are in all likelihood three possible causes:

* B-Tree is just too inherently worse here than SnapTreeMap was
* The extra work now done (besides b-tree) during memtable partition update increases the time taken, and hence number of concurrent operations
* One of the many other changes increases the number of concurrent operations (e.g. changes in threading or increased concurrent_writes), or the size of the partitions in a given memtable (e.g. fewer memtable flushes), is increasing the cost of contention

Ruling out the first one is probably most straightforwardly done by swapping back in SnapTreeMap in a patch and running your tests.  This wouldn't be trivial, but also should be far from terribly difficult, and would avoid striking around in the dark.

Otherwise, the suggestion I made in CASSANDRA-7546 stands as my preferred solution to this kind of problem: deferred updates under contention.  If a swap of head fails, just tag the update onto a queue, and have all readers merge the queue before responding, with all writers merging all deferred updates along with their own.  Readers may also update the b-tree with the result of their merge of there's no contention.


was (Author: benedict):
So, just to crystalise my thoughts on this.  Given this presents with contention, there are in all likelihood three possible causes:

* B-Tree is just too inherently worse here than SnapTreeMap was
* The extra work now done during memtable partition update increases the time taken, and hence number of concurrent operations
* One of the many other changes increases the number of concurrent operations (e.g. changes in threading or increased concurrent_writes), or the size of the partitions in a given memtable (e.g. fewer memtable flushes), is increasing the cost of contention

Ruling out the first one is probably most straightforwardly done by swapping back in SnapTreeMap in a patch and running your tests.  This wouldn't be trivial, but also should be far from terribly difficult, and would avoid striking around in the dark.

Otherwise, the suggestion I made in CASSANDRA-7546 stands as my preferred solution to this kind of problem: deferred updates under contention.  If a swap of head fails, just tag the update onto a queue, and have all readers merge the queue before responding, with all writers merging all deferred updates along with their own.  Readers may also update the b-tree with the result of their merge of there's no contention.


> Memtable Contention in 2.1
> --------------------------
>
>                 Key: CASSANDRA-12668
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12668
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: sankalp kohli
>
> We added a new Btree implementation in 2.1 which causes write performance to go down in Cassandra if there is  lot of contention in the memtable for a CQL partition. Upgrading a cluster from 2.0 to 2.1 with contention causes the cluster to fall apart due to GC. We tried making the defaults added in CASSANDRA-7546 configurable but that did not help. Is there anyway to fix this issue?


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)