cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yabin Meng <yabinm...@gmail.com>
Subject Re: Replacing a dead node in a live Cassandra Cluster
Date Tue, 04 Oct 2016 01:19:58 GMT
Are you sure cassandra.yaml file of the new node is correctly configured?
What is your seeds and listen_address setup of your new node and existing
nodes?

Yabin

On Fri, Sep 30, 2016 at 7:56 PM, Rajath Subramanyam <rajath26@gmail.com>
wrote:

> Hello Cassandra-users,
>
> I was running some tests today. My end goal was to learn more about
> replacing a dead node in a live Cassandra cluster with minimal disruption
> to the existing cluster and figure out a better and faster way of doing the
> same.
>
> I am running a package installation of the following version of Cassandra.
>
> [centos@rj-cassandra-1 testcf-97896450869d11e6a84c4381bf5c5035]$ nodetool
> version
> ReleaseVersion: 2.1.12
>
> I setup a 4 node Cassandra in the lab. I got one non-seed node (lets say
> node1) down by issuing a 'sudo service cassandra stop'. Then following
> following instructions from this link
> <https://docs.datastax.com/en/cassandra/2.1/cassandra/operations/opsReplaceNode.html>,
> I tried to replace node1 with the JMX option -Dcassandra.replace_address=<ip_address_of_node1>.
> However, when I do this the bootstrap fails with the following error in the
> log:
>
> ERROR [main] 2016-09-30 23:54:17,104 CassandraDaemon.java:579 - Exception
> encountered during startup
> java.lang.RuntimeException: Unable to gossip with any seeds
> at org.apache.cassandra.gms.Gossiper.doShadowRound(Gossiper.java:1337)
> ~[apache-cassandra-2.1.12.jar:2.1.12]
> at org.apache.cassandra.service.StorageService.prepareReplacementInfo(StorageService.java:512)
> ~[apache-cassandra-2.1.12.jar:2.1.12]
> at org.apache.cassandra.service.StorageService.prepareToJoin(StorageService.java:783)
> ~[apache-cassandra-2.1.12.jar:2.1.12]
> at org.apache.cassandra.service.StorageService.initServer(StorageService.java:721)
> ~[apache-cassandra-2.1.12.jar:2.1.12]
> at org.apache.cassandra.service.StorageService.initServer(StorageService.java:612)
> ~[apache-cassandra-2.1.12.jar:2.1.12]
> at org.apache.cassandra.service.CassandraDaemon.setup(CassandraDaemon.java:387)
> [apache-cassandra-2.1.12.jar:2.1.12]
> at org.apache.cassandra.service.CassandraDaemon.activate(CassandraDaemon.java:562)
> [apache-cassandra-2.1.12.jar:2.1.12]
> at org.apache.cassandra.service.CassandraDaemon.main(CassandraDaemon.java:651)
> [apache-cassandra-2.1.12.jar:2.1.12]
> WARN  [StorageServiceShutdownHook] 2016-09-30 23:54:17,109
> Gossiper.java:1454 - No local state or state is in silent shutdown, not
> announcing shutdown
> INFO  [StorageServiceShutdownHook] 2016-09-30 23:54:17,109
> MessagingService.java:734 - Waiting for messaging service to quiesce
> INFO  [ACCEPT-/10.7.0.232] 2016-09-30 23:54:17,110
> MessagingService.java:1018 - MessagingService has terminated the accept()
> thread
>
> How do I recover from this error message ?
>
> ------------------------
> Rajath Subramanyam
>
>

Mime
View raw message