incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ryan Hadley <r...@sgizmo.com>
Subject Nodetool removetoken taking days to run.
Date Wed, 14 Sep 2011 13:54:02 GMT
Hi,

So, here's the backstory:

We were running Cassandra 0.7.4 and at one point in time had a node in the ring at 10.84.73.18.
We removed this node from the ring successfully in 0.7.4. It stopped showing in the nodetool
ring command. But occasionally we'd still get weird log entries about failing to write/read
to IP 10.84.73.18.

We upgraded to Cassandra 0.8.4. Now, nodetool ring shows this old node:

10.84.73.18     datacenter1 rack1       Down   Leaving ?               6.71%   32695837177645752437561450928649262701
     

So I started a nodetool removetoken on 32695837177645752437561450928649262701 last Friday.
It's still going strong this morning, on day 5:

./bin/nodetool -h 10.84.73.47 -p 8080 removetoken status
RemovalStatus: Removing token (32695837177645752437561450928649262701). Waiting for replication
confirmation from [/10.84.73.49,/10.84.73.48,/10.84.73.51].

Should I just be patient? Or is something really weird with this node?

Thanks-
ryan
Mime
View raw message