incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Narendra Sharma <narendra.sha...@gmail.com>
Subject Re: Cassandra 1.1.6 - New node bootstrap not completing
Date Fri, 01 Nov 2013 16:36:30 GMT
I was successfully able to bootstrap the node. The issue was RF > 2. Thanks
again Robert.


On Wed, Oct 30, 2013 at 10:29 AM, Narendra Sharma <narendra.sharma@gmail.com
> wrote:

> Thanks Robert.
>
> I didn't realize that some of the keyspaces (not all and esp. the biggest
> one I was focusing on) had RF > 2. I wasted 3 days on it. Thanks again for
> the pointers. I will try again and share the results.
>
>
> On Wed, Oct 30, 2013 at 12:28 AM, Robert Coli <rcoli@eventbrite.com>wrote:
>
>> On Tue, Oct 29, 2013 at 11:45 AM, Narendra Sharma <
>> narendra.sharma@gmail.com> wrote:
>>
>>> We had a cluster of 4 nodes in AWS. The average load on each node was
>>> approx 750GB. We added 4 new nodes. It is now more than 30 hours and the
>>> node is still in JOINING mode.
>>> Specifically I am analyzing the one with IP 10.3.1.29. There is no
>>> compaction or streaming or index building happening.
>>>
>>
>> If your cluster has RF>2, you are bootstrapping two nodes into the same
>> range simultaneously. That is not supported. [1,2] The node you are having
>> the problem with is in the range that is probably overlapping.
>>
>> If I were you I would :
>>
>> 1) stop all "Joining" nodes and wipe their state including system keyspace
>> 2) optionally "removetoken" any nodes which remain in cluster gossip
>> state after stopping
>> 3) re-start/bootstrap them one at a time, waiting for each to complete
>> bootstrapping before starting the next  one
>> 4) (unrelated) Upgrade from 1.1.6 to the head of 1.1.x ASAP.
>>
>> =Rob
>> [1] https://issues.apache.org/jira/browse/CASSANDRA-2434
>> [2]
>> https://issues.apache.org/jira/browse/CASSANDRA-2434?focusedCommentId=13091851&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13091851
>>
>
>
>
> --
> Narendra Sharma
> Software Engineer
> *http://www.aeris.com*
> *http://narendrasharma.blogspot.com/*
>
>


-- 
Narendra Sharma
Software Engineer
*http://www.aeris.com*
*http://narendrasharma.blogspot.com/*

Mime
View raw message