Are you sure cassandra.yaml file of the new node is correctly configured? What is your seeds and listen_address setup of your new node and existing nodes?


On Fri, Sep 30, 2016 at 7:56 PM, Rajath Subramanyam <> wrote:
Hello Cassandra-users, 

I was running some tests today. My end goal was to learn more about replacing a dead node in a live Cassandra cluster with minimal disruption to the existing cluster and figure out a better and faster way of doing the same. 

I am running a package installation of the following version of Cassandra. 

[centos@rj-cassandra-1 testcf-97896450869d11e6a84c4381bf5c5035]$ nodetool version
ReleaseVersion: 2.1.12

I setup a 4 node Cassandra in the lab. I got one non-seed node (lets say node1) down by issuing a 'sudo service cassandra stop'. Then following following instructions from this link, I tried to replace node1 with the JMX option -Dcassandra.replace_address=<ip_address_of_node1>. However, when I do this the bootstrap fails with the following error in the log: 

ERROR [main] 2016-09-30 23:54:17,104 - Exception encountered during startup
java.lang.RuntimeException: Unable to gossip with any seeds
at org.apache.cassandra.gms.Gossiper.doShadowRound( ~[apache-cassandra-2.1.12.jar:2.1.12]
at org.apache.cassandra.service.StorageService.prepareReplacementInfo( ~[apache-cassandra-2.1.12.jar:2.1.12]
at org.apache.cassandra.service.StorageService.prepareToJoin( ~[apache-cassandra-2.1.12.jar:2.1.12]
at org.apache.cassandra.service.StorageService.initServer( ~[apache-cassandra-2.1.12.jar:2.1.12]
at org.apache.cassandra.service.StorageService.initServer( ~[apache-cassandra-2.1.12.jar:2.1.12]
at org.apache.cassandra.service.CassandraDaemon.setup( [apache-cassandra-2.1.12.jar:2.1.12]
at org.apache.cassandra.service.CassandraDaemon.activate( [apache-cassandra-2.1.12.jar:2.1.12]
at org.apache.cassandra.service.CassandraDaemon.main( [apache-cassandra-2.1.12.jar:2.1.12]
WARN  [StorageServiceShutdownHook] 2016-09-30 23:54:17,109 - No local state or state is in silent shutdown, not announcing shutdown
INFO  [StorageServiceShutdownHook] 2016-09-30 23:54:17,109 - Waiting for messaging service to quiesce
INFO  [ACCEPT-/] 2016-09-30 23:54:17,110 - MessagingService has terminated the accept() thread

How do I recover from this error message ? 

Rajath Subramanyam