cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Colby <>
Subject nodetool move trying to stream data to node no longer in cluster
Date Wed, 25 May 2011 12:42:58 GMT
I recently removed a node (with decommission) from our cluster.

I added a couple new nodes and am now trying to rebalance the cluster using nodetool move.

However,  netstats shows that the node being "moved" is trying to stream data to the node
that I already decommissioned yesterday.

The removed node was powered-off, taken out of dns, its IP is not even pingable.   It was
never a seed neither.

This is cassandra 0.7.5 on 64bit linux.   How do I tell the cluster that this node is gone?
 Gossip should have detected this.  The ring commands shows the correct cluster IPs.

Here is a portion of netstats. is the node which was removed.

Mode: Leaving: streaming data to other nodes
Streaming to: /
         progress=280574376402/12434049900 - 2256%

Note is NOT part of the ring.

Address         Status State   Load            Owns    Token                             
                                                       148873535527910577765226390751398592512   Up     Normal  71.73 GB        12.50%  0                                 
   Up     Normal  109.69 GB       12.50%  21267647932558653966460912964485513216   Up     Leaving 281.95 GB       37.50%  85070591730234615865843651857942052863
     <----- currently being "moved"   Up     Normal  210.77 GB       0.00%   85070591730234615865843651857942052864   Up     Normal  289.59 GB       16.67%  113427455640312821154458202477256070484   Up     Normal  299.87 GB       8.33%   127605887595351923798765477786913079296   Up     Normal  94.99 GB        12.50%  148873535527910577765226390751398592511   Up     Normal  103.01 GB       0.00%   148873535527910577765226390751398592512

View raw message