Return-Path: 
 <commits-return-73204-apmail-cassandra-commits-archive=cassandra.apache.org@cassandra.apache.org>
X-Original-To: apmail-cassandra-commits-archive@www.apache.org
Delivered-To: apmail-cassandra-commits-archive@www.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 1945710734
	for <apmail-cassandra-commits-archive@www.apache.org>;
 Thu, 11 Apr 2013 20:17:19 +0000 (UTC)
Received: (qmail 85842 invoked by uid 500); 11 Apr 2013 20:17:18 -0000
Delivered-To: apmail-cassandra-commits-archive@cassandra.apache.org
Received: (qmail 85812 invoked by uid 500); 11 Apr 2013 20:17:18 -0000
Mailing-List: contact commits-help@cassandra.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:commits-help@cassandra.apache.org>
List-Unsubscribe: <mailto:commits-unsubscribe@cassandra.apache.org>
List-Post: <mailto:commits@cassandra.apache.org>
List-Id: <commits.cassandra.apache.org>
Reply-To: dev@cassandra.apache.org
Delivered-To: mailing list commits@cassandra.apache.org
Received: (qmail 85770 invoked by uid 99); 11 Apr 2013 20:17:18 -0000
Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28)
    by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 11 Apr 2013 20:17:18 +0000
Date: Thu, 11 Apr 2013 20:17:18 +0000 (UTC)
From: "Brandon Williams (JIRA)" <jira@apache.org>
To: commits@cassandra.apache.org
Message-ID: <JIRA.12642102.1365705853790.151652.1365711438350@arcas>
In-Reply-To: <JIRA.12642102.1365705853790@arcas>
References: <JIRA.12642102.1365705853790@arcas>
Subject: [jira] [Updated] (CASSANDRA-5456) Large number of bootstrapping
 nodes cause gossip to stop working
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394


     [ https://issues.apache.org/jira/browse/CASSANDRA-5456?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Brandon Williams updated CASSANDRA-5456:
----------------------------------------

    Reviewer: brandon.williams
    
> Large number of bootstrapping nodes cause gossip to stop working
> ----------------------------------------------------------------
>
>                 Key: CASSANDRA-5456
>                 URL: https://issues.apache.org/jira/browse/CASSANDRA-5456
>             Project: Cassandra
>          Issue Type: Bug
>          Components: Core
>    Affects Versions: 1.1.10
>            Reporter: Oleg Kibirev
>         Attachments: PendingRangeCalculatorService.patch
>
>
> Long running section of code in PendingRangeCalculatorService is synchronized on bootstrapTokens. This causes gossip to stop working as it waits for the same lock when a large number of nodes (hundreds in our case) are bootstrapping. Consequently, the whole cluster becomes non-functional. 
> I experimented with the following change in PendingRangeCalculatorService.java and it resolved the problem in our case. Prior code had synchronized around the for loop.
> synchronized(bootstrapTokens) {
>     bootstrapTokens = new LinkedHashMap<Token, InetAddress>(bootstrapTokens);
> }
> for (Map.Entry<Token, InetAddress> entry : bootstrapTokens.entrySet())
> {
>    InetAddress endpoint = entry.getValue();
>    allLeftMetadata.updateNormalToken(entry.getKey(), endpoint);
>    for (Range<Token> range : strategy.getAddressRanges(allLeftMetadata).get(endpoint))
>    pendingRanges.put(range, endpoint);
>    allLeftMetadata.removeEndpoint(endpoint);
> }
>  

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira