cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stu Hood (JIRA)" <j...@apache.org>
Subject [jira] Commented: (CASSANDRA-658) Hinted Handoff CF contention
Date Thu, 31 Dec 2009 19:45:29 GMT

    [ https://issues.apache.org/jira/browse/CASSANDRA-658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12795653#action_12795653
] 

Stu Hood commented on CASSANDRA-658:
------------------------------------

Brilliant... I really like this change.

I think the main reason that we had an Atomic variable for size() in CF and SC was to perform
this calculation, so perhaps that code should be considered dead, and removed. I noticed a
bunch of subtle bugs in it last time I was looking anyway. The size() method can remain, and
calculate the size for each call?

> Hinted Handoff CF contention
> ----------------------------
>
>                 Key: CASSANDRA-658
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-658
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 0.5
>         Environment: debian lenny amd64 OpenJDK 64-Bit Server VM (build 1.6.0_0-b11,
mixed mode)
>            Reporter: Brandon Williams
>            Assignee: Jonathan Ellis
>             Fix For: 0.9
>
>         Attachments: 0001-use-throughput-and-op-count-instead-of-size-and-column.txt,
0002-replace-sharded-row-locks-with-column-level-locking.txt
>
>
> Hinted handoff causes a lot of contention on the HH CF, causing insert speed to massively
drop.  Most of the row mutation stage threads end up blocking on each other at Memtable.resolve.
 This is because HH sends the hint to the closest node, which will always be the node handling
the write.
> To reproduce: start a cluster with even InitialTokens, and begin a constant stream of
writes to one node, with an even key distribution. (I used 4 nodes and stress.py in random
mode.)  Take a node down, and the insert rate begin to drop, eventually settling between 100-300/s
and sustaining there.  Bringing the down node back up will restore the original insert rate.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message