Thanks Alain, this seems to have stopped the log messages, even though nodetool gossipinfo still shows all the old nodes there. (And now all sharing token 50? I dunno where that came from.) Will they eventually fall away from the cluster, or are they there for good?On Mar 21, 2013, at 11:53 AM, Alain RODRIGUEZ wrote:Using the unsafeAssassinateEndpoint function with old IPs from JMX should do the trick.
This was already discussed in this mailing list, search using "unsafeAssassinateEndpoint" as keyword to know all that you need to know about it.Hope you'll be ok after that.Alain2013/3/21 Ben Chobot <email@example.com>I've got a 1.1.5 cluster, and a few weeks ago I removed some nodes from it. (I was trying to upgrade nodes from AWS' large to xlarge, and for some reason that made sense at the time, it seemed better to double my nodes and then decommission the smaller ones, rather than to simply rebuild the existing nodes serially.)
Now the remaining nodes are all frequently logging that the old, decommissioned nodes are dead and that their old token is being removed.... which is great, I guess, but why does my cluster know about them at all? Doing a nodetool removetoken doesn't work, as the dead nodes don't display in the ring. Is this expected behavior after a nodetool decommission? Is maybe something cached that I can safely uncache?