incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Sameer Farooqui <cassandral...@gmail.com>
Subject Re: EC2 node adding trouble
Date Tue, 24 May 2011 21:58:34 GMT
Even with AutoBootstrap it is recommended that you always specify
the InitialToken on the new node because the picking of an initial token
will almost certainly result in an unbalanced ring.

Right now, I'm afraid that if you simply copied the YAML file from one of
the two nodes to the 3rd node, then the 3rd node has an incorrect
initial_token setting (it may be conflicting with node 1 or 2's range).

Some info about how to choose tokens:
http://wiki.apache.org/cassandra/Operations#Token_selection
<http://wiki.apache.org/cassandra/Operations#Token_selection>
http://journal.paul.querna.org/articles/2010/09/24/cassandra-token-selection/

So, once you know what token each of the 3 nodes should have, shut down the
first two nodes, change their tokens and add the correct token to the 3rd
node (in the YAML file).

If this doesn't fix it, start to look at the Cassandra log file to
troubleshoot. It's located usually at /var/log/cassandra and is by default
only logging at INFO level. You can make it more granular by changing the
logging level in the file <cassandra-home>/conf/log4j-server.properties and
change the line with INFO in it to:
log4j.rootLogger=DEBUG,stdout,R

Note that this will slow things down performance-wise in the cluster since
every I/O is logged. But it really helps with learning how Cassandra works
and with troubleshooting.

research links:
http://wiki.apache.org/cassandra/FAQ#existing_data_when_adding_new_nodes



On Tue, May 24, 2011 at 12:17 PM, Marcus Bointon
<marcus@synchromedia.co.uk>wrote:

> On 24 May 2011, at 19:33, Sameer Farooqui wrote:
>
> > What region and availability zones are the different nodes in? Are you
> using EC2 Snitch? Did you set up the cluster using the Datastax AMI?
>
> The two existing ones are in us-east-1c and us-east-1d, the new one is in
> us-east-1c, so all same region, different AZs. I'm using the Alestic Ubuntu
> AMIs. I've not heard of either EC2 snitch or Datastax. Are they particularly
> suited to cassandra?
>
> Marcus

Mime
View raw message