incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Coli <rc...@eventbrite.com>
Subject Re: One of my nodes is in the wrong datacenter - help!
Date Tue, 04 Feb 2014 00:30:52 GMT
On Sun, Feb 2, 2014 at 10:48 AM, Sholes, Joshua <
Joshua_Sholes@cable.comcast.com> wrote:

>  I had a node in my 8-node production 1.2.8 cluster have a serious
> problem and need to be removed and rebuilt.   However, after doing nodetool
> removenode and then bootstrapping a new node on the same IP address, the
> new node somehow ended up with a different datacenter name (the rest of the
> nodes are in dc $NAME, and the new one is in dc $NAME6934724 -- as in, a
> string of seemingly random numbers appended to the correct name).   How can
> I force it to change DC names back to what it should be?
>

You could change the entry in the system.local columnfamily on the affected
node...

cqlsh > update system.local set data_center = "$NAME";

... but that is Not Supported and may have side effects of which I am not
aware.

 I'm working with 500+GB per node here so bootstrapping it again is not a
> huge issue, but I'd prefer to avoid it anyway.  I am NOT able to change the
> node's IP address at this time so I'm stuck with bootstrapping a new node
> in the same place, which my gut feeling tells me might be part of the
> problem.
>

Note that replace_node/replace_token are broken in 1.2.8, did you attempt
to use either of these? I presume not because you said you did removenode...

 If I were you, I would probably removenode and re-bootstrap, as the safest
alternative.

As an aside, while trying to deal with this issue you should be aware of
this ticket, so you do not do the sequence of actions it describes.

https://issues.apache.org/jira/browse/CASSANDRA-6615

=Rob

Mime
View raw message