cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Len Bucchino <>
Subject RE: New node not joining
Date Thu, 05 May 2011 18:24:48 GMT
Adding the fourth node to the cluster with an empty schema using auto_bootstrap was not successful.
 A nodetool netstats on the new node shows "Mode: Joining: getting bootstrap token" similar
to what the third node did before it was manually added.  Also, there are no exceptions in
the logs but it never joins the ring.

From: Sanjeev Kulkarni []
Sent: Thursday, May 05, 2011 11:47 AM
Subject: Re: New node not joining

Hi Len,
This looks like a decent workaround. I would be very interested to see how the addition of
the 4th node went. Please post it whenever you get a chance.

On Thu, May 5, 2011 at 6:47 AM, Len Bucchino <<>>
I have the same problem on 0.7.5 auto bootstrapping a 3rd node onto an empty 2 node test cluster
(the two nodes were manually added) and the it currently has an empty schema.  My log entries
look similar to yours.  I took the new token it says its going to use from the log file added
it to the yaml and turned off auto bootstrap and the node added fine.  I'm bringing up a 4th
node now and will see if it has the same problem auto bootstrapping.

From: Sanjeev Kulkarni [<>]
Sent: Thursday, May 05, 2011 2:18 AM
Subject: New node not joining
Hey guys,
I'm running into what seems like a very basic problem.
I have a one node cassandra instance. Version 0.7.5. Freshly installed. Contains no data.
The cassandra.yaml is the same as the default one that is supplied, except for data/commitlog/saved_caches
I also changed the addresses to point to a externally visible ip address.
The cassandra comes up nicely and is ready to accept thrift connections.
I do a nodetool and this is what I get.  Up     Normal  6.54 KB         100.00% 110022862993086789903543147927259579701

Which seems right to me.

Now I start another node. Almost identical configuration to the first one. Except the bootstrap
is turned true and seeds appropriately set.
When I start the second, I notice that the second one contacts the first node to get the new
I see the following lines in the first machine(the seed machine).

INFO [GossipStage:1] 2011-05-05 07:00:20,427 (line 628) Node /<>
has restarted,
now UP again
 INFO [HintedHandoff:1] 2011-05-05 07:00:55,162 (line 304) Started
hinted handoff for endpoint /<>
 INFO [HintedHandoff:1] 2011-05-05 07:00:55,164 (line 360) Finished
hinted hand
off of 0 rows to endpoint /<>

However when i do a node ring, I still get  Up     Normal  6.54 KB         100.00% 110022862993086789903543147927259579701

Even though the second node has come up. On the second machine the logs say

INFO [main] 2011-05-05 07:00:19,124 (line 504) Joining: getting load information
 INFO [main] 2011-05-05 07:00:19,124 (line 351) Sleeping 90000 ms
to wait for load information...
 INFO [GossipStage:1] 2011-05-05 07:00:20,828 (line 628) Node /<>
has restarted, now UP again
 INFO [HintedHandoff:1] 2011-05-05 07:00:29,548 (line 304) Started
hinted handoff for endpoint /<>
 INFO [HintedHandoff:1] 2011-05-05 07:00:29,550 (line 360) Finished
hinted handoff of 0 rows to endpoint /<>
 INFO [main] 2011-05-05 07:01:49,137 (line 504) Joining: getting bootstrap
 INFO [main] 2011-05-05 07:01:49,148 (line 148) New token will be 24952271262852174037699496069317526837
to assume load from /<>
 INFO [main] 2011-05-05 07:01:49,150 (line 72) Will not load MX4J, mx4j-tools.jar
is not in the classpath
 INFO [main] 2011-05-05 07:01:49,259 (line 112) Binding thrift service
to /<>
 INFO [main] 2011-05-05 07:01:49,262 (line 126) Using TFastFramedTransport
with a max frame size of 15728640 bytes.
 INFO [Thread-5] 2011-05-05 07:01:49,266 (line 154) Listening for thrift

This seems to indicate that the second node has joined the ring. And has gotten its key range.
Am I missing anything?


View raw message