incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dimitry Lvovsky <dimi...@reviewpro.com>
Subject New nodes won't bootstrap on .66
Date Tue, 26 Oct 2010 09:42:57 GMT
Hi All,
We recently upgraded from .65 to .66 after which we tried adding a new node to our cluster.
 We left it bootstrapping and after 3 days, it still refused to join the ring.  The strange
thing is that  nodetool info shows  50GB of load and  nodetool ring shows that it sees the
rest of ring, which it is not part of.  We tried the process again with another server --
again the same thing as before:


//from machine 192.168.2.18

 /opt/cassandra/bin/nodetool -h localhost -p 8999 info
131373516047318302934572185119435768941
Load             : 52.85 GB
Generation No    : 1287761987
Uptime (seconds) : 323157
Heap Memory (MB) : 795.42 / 1945.63


 /opt/cassandra/bin/nodetool -h localhost -p 8999 ring
Address       Status     Load          Range                                      Ring
                                       158573510920250391466717289405976537674    
192.168.2.22  Up         59.45 GB      28203205416427384773583427414698832202     |<--|
192.168.2.23  Up         44.95 GB      60562227403709245514637766500430120055     |   |
192.168.2.20  Up         47.15 GB      104160057322065544623939416372654814065    |   |
192.168.2.21  Up         61.04 GB      158573510920250391466717289405976537674    |-->|

opt/cassandra/bin/nodetool -h localhost -p 8999 streams
Mode: Bootstrapping
Not sending any streams.
Not receiving any streams.


Whats more, while looking at the log of one of the nodes I see gossip messages from 192.168.1.17
-- the first node we tried to add to the cluster but which is not running at the the time
of the log message:
 INFO [Timer-0] 2010-10-26 02:13:20,340 Gossiper.java (line 406) FatClient /192.168.2.17 has
been silent for 3600000ms, removing from gossip
 INFO [GMFD:1] 2010-10-26 02:13:51,398 Gossiper.java (line 591) Node /192.168.2.17 is now
part of the cluster


Thanks in advance for the help,
Dimitry  
Mime
View raw message