cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Benedict (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (CASSANDRA-12668) Memtable Contention in 2.1
Date Tue, 20 Sep 2016 09:25:20 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-12668?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15506130#comment-15506130
] 

Benedict edited comment on CASSANDRA-12668 at 9/20/16 9:24 AM:
---------------------------------------------------------------

So, just to crystalise my thoughts on this.  Given this presents with contention, there are
in all likelihood three possible causes:

* B-Tree is just too inherently worse here than SnapTreeMap was
* The extra work now done (besides b-tree) during memtable partition update increases the
time taken, and hence number of concurrent operations
* One of the many other changes increases the number of concurrent operations (e.g. changes
in threading or increased concurrent_writes), or the size of the partitions in a given memtable
(e.g. fewer memtable flushes), is increasing the cost of contention

Ruling out the first one is probably most straightforwardly done by swapping back in SnapTreeMap
in a patch and running your tests.  This wouldn't be trivial, but also should be far from
terribly difficult, and would avoid striking around in the dark.

Otherwise, the suggestion I made in CASSANDRA-7546 stands as my preferred solution to this
kind of problem: deferred updates under contention.  If a swap of head fails, just tag the
update onto a queue, and have all readers merge the queue before responding, with all writers
merging all deferred updates along with their own.  Readers may also update the b-tree with
the result of their merge of there's no contention.



was (Author: benedict):
So, just to crystalise my thoughts on this.  Given this presents with contention, there are
in all likelihood three possible causes:

* B-Tree is just too inherently worse here than SnapTreeMap was
* The extra work now done during memtable partition update increases the time taken, and hence
number of concurrent operations
* One of the many other changes increases the number of concurrent operations (e.g. changes
in threading or increased concurrent_writes), or the size of the partitions in a given memtable
(e.g. fewer memtable flushes), is increasing the cost of contention

Ruling out the first one is probably most straightforwardly done by swapping back in SnapTreeMap
in a patch and running your tests.  This wouldn't be trivial, but also should be far from
terribly difficult, and would avoid striking around in the dark.

Otherwise, the suggestion I made in CASSANDRA-7546 stands as my preferred solution to this
kind of problem: deferred updates under contention.  If a swap of head fails, just tag the
update onto a queue, and have all readers merge the queue before responding, with all writers
merging all deferred updates along with their own.  Readers may also update the b-tree with
the result of their merge of there's no contention.


> Memtable Contention in 2.1
> --------------------------
>
>                 Key: CASSANDRA-12668
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-12668
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: sankalp kohli
>
> We added a new Btree implementation in 2.1 which causes write performance to go down
in Cassandra if there is  lot of contention in the memtable for a CQL partition. Upgrading
a cluster from 2.0 to 2.1 with contention causes the cluster to fall apart due to GC. We tried
making the defaults added in CASSANDRA-7546 configurable but that did not help. Is there anyway
to fix this issue?



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message