cassandra-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Ellis (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (CASSANDRA-7758) Some gossip messages are very slow to process on vnode clusters
Date Tue, 12 Aug 2014 18:59:11 GMT

     [ https://issues.apache.org/jira/browse/CASSANDRA-7758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Jonathan Ellis updated CASSANDRA-7758:
--------------------------------------

         Reviewer: Jonathan Ellis
    Reproduced In: 2.0.9, 1.2.18  (was: 1.2.18, 2.0.9)

> Some gossip messages are very slow to process on vnode clusters
> ---------------------------------------------------------------
>
>                 Key: CASSANDRA-7758
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-7758
>             Project: Cassandra
>          Issue Type: Bug
>            Reporter: Rick Branson
>            Assignee: Rick Branson
>         Attachments: 7758-1-v1.2.txt, 7758-1-v2.0.txt
>
>
> Both StorageService.getChangedRangesForLeaving and PendingRangeCalculatorService.calculatePendingRanges
clone the TokenMetadata map once per range that a node covers. This is a very expensive operation
on a cluster with vnodes since the TokenMetadata map can easily be tens of thousands of entries,
even for a small cluster, and this runs thousands of times for each changing node. This can
cause big cluster-wide slowdowns & problems in the GossipStage and PendingRangeCalculator
thread pools. Processing just one message can generate enough garbage to pause the entire
cluster simultaneously when one of these is being propagated. Patches forthcoming.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message