cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Jirsa <>
Subject Re: Nodes show different number of tokens than initially
Date Wed, 31 Jan 2018 16:17:57 GMT

> On Jan 31, 2018, at 12:35 AM, Oleksandr Shulgin <>
>> On Tue, Jan 30, 2018 at 5:44 PM, Jeff Jirsa <> wrote:
>> All DCs in a cluster use the same token space in the DHT,
> I can't believe my bloody eyes, but this seems to be true...

I don’t know why this is a surprise (maybe because people like to talk about multiple rings,
but the fact that replication strategy is set per keyspace and that you could use SimpleStrategy
in a multiple dc cluster demonstrates this), but we can chat about that another time 

>> so token conflicts across datacenters are invalid config
> If this is deemed invalid config why does the new node *silently* steals the existing
token, badly affecting the ownership of the rest of the nodes?  It should just refuse to start!

Philosophically, With multiple DCs, it may start up and not see the other DC for minutes/hours/days
before it realizes there’s a token conflict - what should it do then? Which node gets stopped?
If your suggestion to resolve that is to make sure we see the whole ring before starting up,
we end up in a situation where we try not to startup unless we can see all nodes, and create
outages during DC separations. 

Distributed systems and occasional availability make these decisions harder.

Please open a jira if you think it’s wrong, but I’m not sure I know what the “right”
answer is either.
View raw message