cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Curious Patient <>
Subject Re: problem removing dead node from ring
Date Sun, 08 Jun 2014 01:18:26 GMT
Hey all,

 OK I gave removing the downed node from the cassandra ring another try.

To recap what's going on, this is what my ring looks like with nodetool

[root@beta-new:~] #nodetool status

Datacenter: datacenter1



|/ State=Normal/Leaving/Joining/Moving

--  Address         Load       Tokens  Owns   Host ID

UN  178.38 KB  256     49.4%
fd2f76ae-8dcf-4e93-a37f-bf1e9088696e  rack1

DN     ?          256     50.6%
f2a48fc7-a362-43f5-9061-4bb3739fdeaf  rack1

So I followed the steps in this document one more time:

And setup the following in the cassandra.yaml according to the above

cluster_name: ‘Test Cluster'

num_tokens: 256



auto_bootstrap: yes


endpoint_snitch: SimpleSnitch

initial_token: -9173731940639284976

The initial_token is the one belonging to the dead node that I'm trying to
get rid of.

I then make sure that the /var/lib/casssandra directory is completely empty
and run this startup command:

[root@cassandra1 cassandrahome]# ./bin/cassandra
-Dcassandra.replace_address= -f

Using the IP of the node I want to remove as the value to

And when I do this is the error I get:

java.lang.RuntimeException: Cannot replace_address / because it
doesn't exist in gossip

So how can I get cassandra to realize that this node needs to be replaced
and that it SHOULDN'T exist in gossip because the node is down? That would
seem obvious to me, so why isn't it obvious to her? :)



On Wed, Jun 4, 2014 at 4:36 PM, Robert Coli <> wrote:

> On Tue, Jun 3, 2014 at 9:03 PM, Matthew Allen <>
> wrote:
>> Thanks Robert, this makes perfect sense.  Do you know if CASSANDRA-6961
>> will be ported to 1.2.x ?
> I just asked driftx, he said "not gonna happen."
>> And apologies if these appear to be dumb questions, but is a repair more
>> suitable than a rebuild because the rebuild only contacts 1 replica (per
>> range), which may itself contain stale data ?
> Exactly that.
> Discusses related issues in quite some detail. The tl;dr is that until
> 2434 is resolved, streams do not necessarily come from the node departing
> the range, and therefore the "unique replica count" is decreased by
> changing cluster topology.
> =Rob

View raw message