cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Durity, Sean R" <SEAN_R_DUR...@homedepot.com>
Subject RE: [EXTERNAL] Re: does c* 3.0 use one ring for all datacenters?
Date Thu, 12 Apr 2018 20:15:27 GMT
Perhaps it would help to also note that every DC has a full copy (or more) of the data. Data
for a given token exists on all DCs (assuming the keyspace is replicated to all DCs). A given
node’s token ownership just determines where that data goes *for that DC*


Sean Durity
From: Jeff Jirsa [mailto:jjirsa@gmail.com]
Sent: Wednesday, April 11, 2018 1:55 PM
To: cassandra <user@cassandra.apache.org>
Subject: [EXTERNAL] Re: does c* 3.0 use one ring for all datacenters?

The concept of DCs is just a feature offered by one replication strategy on top of the single
ring.

What would multiple rings look like for a non-NTS replication strategy? It would become meaningless
and broken.

On Wed, Apr 11, 2018 at 10:33 AM, Jinhua Luo <luajit.io@gmail.com<mailto:luajit.io@gmail.com>>
wrote:
Even under one ring, when you write a key, you need to iterate the DC
list declared in the snith.
And for each iteration, you need to skip token ranges not owned by that DC.
So I could not figure out the rationality to mix DCs into one ring.

However, if each DC has its own ring, I could repeat SimpleStrategy
procedure on each DC, isn't that simpler?

2018-04-12 1:13 GMT+08:00 Jeff Jirsa <jjirsa@gmail.com<mailto:jjirsa@gmail.com>>:
> It's probably mostly a carry-over from dynamo paper.
>
> I'm skeptical of your claim that N token rings is somehow easier than 1,
> especially given the lack of familiarity with the codebase to know how many
> rings are involved.
>
>
>
> On Wed, Apr 11, 2018 at 10:07 AM, Jinhua Luo <luajit.io@gmail.com<mailto:luajit.io@gmail.com>>
wrote:
>>
>> What's the benefits of one ring?
>> It seems that separate rings could archive the same goal, and make
>> architecture simpler.
>>
>> 2018-04-12 0:59 GMT+08:00 Jeff Jirsa <jjirsa@gmail.com<mailto:jjirsa@gmail.com>>:
>> > When you add DC3, they'll get tokens (that aren't currently in use in
>> > any
>> > existing DC). Either you assign tokens (let's pretend we manually
>> > assigned
>> > the other ones, since DC2 = DC1 + 1), but cassandra can also
>> > auto-calculate
>> > them, the exact behavior of which varies by version.
>> >
>> >
>> > Let's pretend it's old style random assignment, and we end up with DC3
>> > having 4, 17, 22, 36, 48, 53, 64, 73, 83
>> >
>> > In this case:
>> >
>> > If you use SimpleStrategy and RF=3, a key with token 5 would be placed
>> > on
>> > the hosts with token 10, 11, 17
>> > If you use NetworkTopologyStrategy with RF=3 per DC, a key with token 5
>> > would be placed on the hosts with tokens 10,20,30 ; 11, 21,31 ; 17, 22,
>> > 36
>> >
>> >
>> >
>> >
>> > On Wed, Apr 11, 2018 at 9:36 AM, Jinhua Luo <luajit.io@gmail.com<mailto:luajit.io@gmail.com>>
wrote:
>> >>
>> >> What if I add a new DC3?
>> >> The token ranges would reshuffled into DC1, DC2, DC3?
>> >>
>> >> 2018-04-11 22:06 GMT+08:00 Jeff Jirsa <jjirsa@gmail.com<mailto:jjirsa@gmail.com>>:
>> >> > Confirming again that it's definitely one ring.
>> >> >
>> >> > DC1 may have tokens 0, 10, 20, 30, 40, 50, 60, 70, 80
>> >> > DC2 may have tokens 1, 11, 21, 31, 41, 51, 61, 71, 81
>> >> >
>> >> > If you use SimpleStrategy and RF=3, a key with token 5 would be
>> >> > placed
>> >> > on
>> >> > the hosts with token 10, 11, 20
>> >> > If you use NetworkTopologyStrategy with RF=3 per DC, a key with token
>> >> > 5
>> >> > would be placed on the hosts with tokens 10,20,30 and 11, 21,31
>> >> >
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > On Wed, Apr 11, 2018 at 6:27 AM, Jinhua Luo <luajit.io@gmail.com<mailto:luajit.io@gmail.com>>
>> >> > wrote:
>> >> >>
>> >> >> Is it a different answer? One ring?
>> >> >>
>> >> >> Could you explain your answer according to my example?
>> >> >>
>> >> >> 2018-04-11 21:24 GMT+08:00 Jonathan Haddad <jon@jonhaddad.com<mailto:jon@jonhaddad.com>>:
>> >> >> > There has always been a single ring.
>> >> >> >
>> >> >> > You can specify how many nodes in each DC you want and it’ll
>> >> >> > figure
>> >> >> > out
>> >> >> > how
>> >> >> > to do it as long as you have the right snitch and are using
>> >> >> > NetworkToploogyStrategy.
>> >> >> >
>> >> >> >
>> >> >> > On Wed, Apr 11, 2018 at 6:11 AM Jinhua Luo <luajit.io@gmail.com<mailto:luajit.io@gmail.com>>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> Let me clarify my question:
>> >> >> >>
>> >> >> >> Given we have a cluster of two DCs, each DC has 2 nodes,
each
>> >> >> >> node
>> >> >> >> sets num_token as 50.
>> >> >> >> Then how are token ranges distributed in the cluster?
>> >> >> >>
>> >> >> >> If there is one global ring, then it may be (To simply
the case,
>> >> >> >> let's
>> >> >> >> assume vnodes=1):
>> >> >> >> {dc1, node1} 1-50
>> >> >> >> {dc2, node1} 51-100
>> >> >> >> {dc1, node1} 101-150
>> >> >> >> {dc1, node2} 151-200
>> >> >> >>
>> >> >> >> But here comes more questions:
>> >> >> >> a) what if I add a new datacenter? Then the token ranges
need to
>> >> >> >> be
>> >> >> >> re-balanced?
>> >> >> >> If so, what about the data associated with the ranges
to be
>> >> >> >> balanced?
>> >> >> >> move them among DCs?
>> >> >> >> But that doesn't make sense, because each keyspace would
specify
>> >> >> >> its
>> >> >> >> snith and fix the DCs to store then.
>> >> >> >>
>> >> >> >> b) It seems no benefits from same ring, because of the
snith.
>> >> >> >>
>> >> >> >> If each DC has own ring, then it may be:
>> >> >> >> {dc1, node1} 1-50
>> >> >> >> {dc1, node1} 51-100
>> >> >> >> {dc2, node1} 1-50
>> >> >> >> {dc2, node1} 51-100
>> >> >> >>
>> >> >> >> I think this is not a trivial question, because each key
would be
>> >> >> >> hashed to determine the token it belongs to, and
>> >> >> >> the token range distribution in turns determine which
node the
>> >> >> >> key
>> >> >> >> belongs
>> >> >> >> to.
>> >> >> >>
>> >> >> >> Any official answer?
>> >> >> >>
>> >> >> >>
>> >> >> >> 2018-04-11 20:54 GMT+08:00 Jacques-Henri Berthemet
>> >> >> >> <jacques-henri.berthemet@genesys.com<mailto:jacques-henri.berthemet@genesys.com>>:
>> >> >> >> > Maybe I misunderstood something but from what I understand,
>> >> >> >> > each
>> >> >> >> > DC
>> >> >> >> > have
>> >> >> >> > the same ring (0-100 in you example) but it's split
differently
>> >> >> >> > between
>> >> >> >> > nodes in each DC. I think it's the same principle
if using
>> >> >> >> > vnode
>> >> >> >> > or
>> >> >> >> > not.
>> >> >> >> >
>> >> >> >> > I think the confusion comes from the fact that the
ring range
>> >> >> >> > is
>> >> >> >> > the
>> >> >> >> > same (0-100) but each DC manages it differently because
nodes
>> >> >> >> > are
>> >> >> >> > different.
>> >> >> >> >
>> >> >> >> > --
>> >> >> >> > Jacques-Henri Berthemet
>> >> >> >> >
>> >> >> >> > -----Original Message-----
>> >> >> >> > From: Jinhua Luo [mailto:luajit.io@gmail.com<mailto:luajit.io@gmail.com>]
>> >> >> >> > Sent: Wednesday, April 11, 2018 2:26 PM
>> >> >> >> > To: user@cassandra.apache.org<mailto:user@cassandra.apache.org>
>> >> >> >> > Subject: Re: does c* 3.0 use one ring for all datacenters?
>> >> >> >> >
>> >> >> >> > Thanks for your reply. I also think separate rings
are more
>> >> >> >> > reasonable.
>> >> >> >> >
>> >> >> >> > So one ring for one dc is only for c* 1.x or 2.x
without vnode?
>> >> >> >> >
>> >> >> >> > Check these references:
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > https://docs.datastax.com/en/archived/cassandra/1.1/docs/initialize/token_generation.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__docs.datastax.com_en_archived_cassandra_1.1_docs_initialize_token-5Fgeneration.html&d=DwMFaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=lE-camvsUq1ytrS96w9lrdMu6OcCUPGc_DRBi6lug1I&s=MVqypsYVFXNkhOPdG6gRQ1_S4hCvS2oFRn15ZFVSDR0&e=>
>> >> >> >> > http://www.luketillman.com/one-token-ring-to-rule-them-all/<https://urldefense.proofpoint.com/v2/url?u=http-3A__www.luketillman.com_one-2Dtoken-2Dring-2Dto-2Drule-2Dthem-2Dall_&d=DwMFaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=lE-camvsUq1ytrS96w9lrdMu6OcCUPGc_DRBi6lug1I&s=hPiXu7mdI0gMb4yGeFU75J94zJBEivRzE5PyHlsc9CQ&e=>
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > https://community.apigee.com/articles/13096/cassandra-token-distribution.html<https://urldefense.proofpoint.com/v2/url?u=https-3A__community.apigee.com_articles_13096_cassandra-2Dtoken-2Ddistribution.html&d=DwMFaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=lE-camvsUq1ytrS96w9lrdMu6OcCUPGc_DRBi6lug1I&s=3YEEsf6tGrf6PdLZgB5r5jmQ0mTaUMgUbSaZD5P6QwA&e=>
>> >> >> >> >
>> >> >> >> > Even the riak official said c* splits the ring across
dc:
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > http://basho.com/posts/business/riak-vs-cassandra-an-updated-brief-comparison/<https://urldefense.proofpoint.com/v2/url?u=http-3A__basho.com_posts_business_riak-2Dvs-2Dcassandra-2Dan-2Dupdated-2Dbrief-2Dcomparison_&d=DwMFaQ&c=MtgQEAMQGqekjTjiAhkudQ&r=aC_gxC6z_4f9GLlbWiKzHm1vucZTtVYWDDvyLkh8IaQ&m=lE-camvsUq1ytrS96w9lrdMu6OcCUPGc_DRBi6lug1I&s=npfCp7wqcrxtVXNIGRTLx3T5QcDos3wNV7JpibstvEg&e=>
>> >> >> >> >
>> >> >> >> > Why they said each dc has its own ring?
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > 2018-04-11 19:55 GMT+08:00 Jacques-Henri Berthemet
>> >> >> >> > <jacques-henri.berthemet@genesys.com<mailto:jacques-henri.berthemet@genesys.com>>:
>> >> >> >> >> Hi,
>> >> >> >> >>
>> >> >> >> >> Each DC has the whole ring, each DC contains
a copy of the
>> >> >> >> >> same
>> >> >> >> >> data.
>> >> >> >> >> When you add replication to a new DC, all data
is copied to
>> >> >> >> >> the
>> >> >> >> >> new
>> >> >> >> >> DC.
>> >> >> >> >>
>> >> >> >> >> Within a DC, each range of token is 'owned' by
a (primary)
>> >> >> >> >> node
>> >> >> >> >> (and
>> >> >> >> >> replicas if you have RF > 1). If you add/remove
a node in a
>> >> >> >> >> DC,
>> >> >> >> >> tokens will
>> >> >> >> >> be rearranged between all nodes within the DC
only, the other
>> >> >> >> >> DCs
>> >> >> >> >> won't be
>> >> >> >> >> affected.
>> >> >> >> >>
>> >> >> >> >> --
>> >> >> >> >> Jacques-Henri Berthemet
>> >> >> >> >>
>> >> >> >> >> -----Original Message-----
>> >> >> >> >> From: Jinhua Luo [mailto:luajit.io@gmail.com<mailto:luajit.io@gmail.com>]
>> >> >> >> >> Sent: Wednesday, April 11, 2018 12:35 PM
>> >> >> >> >> To: user@cassandra.apache.org<mailto:user@cassandra.apache.org>
>> >> >> >> >> Subject: does c* 3.0 use one ring for all datacenters?
>> >> >> >> >>
>> >> >> >> >> Hi All,
>> >> >> >> >>
>> >> >> >> >> I know it seems a stupid question, but I am really
confused
>> >> >> >> >> about
>> >> >> >> >> the
>> >> >> >> >> documents on the internet related to this topic,
especially it
>> >> >> >> >> seems
>> >> >> >> >> that it
>> >> >> >> >> has different answers for c* with vnodes or not.
>> >> >> >> >>
>> >> >> >> >> Let's assume the token range is 1-100 for the
whole cluster,
>> >> >> >> >> how
>> >> >> >> >> does
>> >> >> >> >> it distributed into the datacenters? Think that
the number of
>> >> >> >> >> datacenters is
>> >> >> >> >> dynamic in a cluster, if there is only one ring,
then the
>> >> >> >> >> token
>> >> >> >> >> range would
>> >> >> >> >> change on each node when I add a new datacenter
into the
>> >> >> >> >> cluster?
>> >> >> >> >> Then it
>> >> >> >> >> would involve data migration? It doesn't make
sense.
>> >> >> >> >>
>> >> >> >> >> Looking forward to clarification for c* 3.0,
thanks!
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> ---------------------------------------------------------------------
>> >> >> >> >> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org<mailto:user-unsubscribe@cassandra.apache.org>
>> >> >> >> >> For additional commands, e-mail:
>> >> >> >> >> user-help@cassandra.apache.org<mailto:user-help@cassandra.apache.org>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> ---------------------------------------------------------------------
>> >> >> >> >> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org<mailto:user-unsubscribe@cassandra.apache.org>
>> >> >> >> >> For additional commands, e-mail:
>> >> >> >> >> user-help@cassandra.apache.org<mailto:user-help@cassandra.apache.org>
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > ---------------------------------------------------------------------
>> >> >> >> > To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org<mailto:user-unsubscribe@cassandra.apache.org>
>> >> >> >> > For additional commands, e-mail: user-help@cassandra.apache.org<mailto:user-help@cassandra.apache.org>
>> >> >> >> >
>> >> >> >>
>> >> >> >>
>> >> >> >>
>> >> >> >> ---------------------------------------------------------------------
>> >> >> >> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org<mailto:user-unsubscribe@cassandra.apache.org>
>> >> >> >> For additional commands, e-mail: user-help@cassandra.apache.org<mailto:user-help@cassandra.apache.org>
>> >> >> >>
>> >> >> >
>> >> >>
>> >> >>
>> >> >> ---------------------------------------------------------------------
>> >> >> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org<mailto:user-unsubscribe@cassandra.apache.org>
>> >> >> For additional commands, e-mail: user-help@cassandra.apache.org<mailto:user-help@cassandra.apache.org>
>> >> >>
>> >> >
>> >>
>> >> ---------------------------------------------------------------------
>> >> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org<mailto:user-unsubscribe@cassandra.apache.org>
>> >> For additional commands, e-mail: user-help@cassandra.apache.org<mailto:user-help@cassandra.apache.org>
>> >>
>> >
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org<mailto:user-unsubscribe@cassandra.apache.org>
>> For additional commands, e-mail: user-help@cassandra.apache.org<mailto:user-help@cassandra.apache.org>
>>
>

---------------------------------------------------------------------
To unsubscribe, e-mail: user-unsubscribe@cassandra.apache.org<mailto:user-unsubscribe@cassandra.apache.org>
For additional commands, e-mail: user-help@cassandra.apache.org<mailto:user-help@cassandra.apache.org>


________________________________

The information in this Internet Email is confidential and may be legally privileged. It is
intended solely for the addressee. Access to this Email by anyone else is unauthorized. If
you are not the intended recipient, any disclosure, copying, distribution or any action taken
or omitted to be taken in reliance on it, is prohibited and may be unlawful. When addressed
to our clients any opinions or advice contained in this Email are subject to the terms and
conditions expressed in any applicable governing The Home Depot terms of business or client
engagement letter. The Home Depot disclaims all responsibility and liability for the accuracy
and content of this attachment and for any damages or losses arising from any inaccuracies,
errors, viruses, e.g., worms, trojan horses, etc., or other items of a destructive nature,
which may be contained in this attachment and shall not be liable for direct, indirect, consequential
or special damages in connection with this e-mail message or its attachment.
Mime
View raw message