incubator-cassandra-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stu Hood" <stu.h...@rackspace.com>
Subject RE: Ring management and load balance
Date Fri, 26 Mar 2010 00:18:58 GMT
It is much more likely that you always increase your cluster in size by a certain large percentage.
With a 10 node cluster, you are likely to add 5 nodes at a time, and with a 100 node cluster
you'll probably add 25 to 50 per batch.


-----Original Message-----
From: "Daniel Kluesing" <dk@bluekai.com>
Sent: Thursday, March 25, 2010 6:59pm
To: "user@cassandra.apache.org" <user@cassandra.apache.org>
Subject: RE: Ring management and load balance

I agree it's only a problem with 'small' clusters - but it seems like 'small' is 'most users'?
Even with 10 nodes it looks like a pretty big imbalance if I add an 11th node, and don't add
the other 9 or move a large part of the ring. Or in practice have folks not had trouble with
incremental scalability?



-----Original Message-----
From: Jonathan Ellis [mailto:jbellis@gmail.com] 
Sent: Thursday, March 25, 2010 2:27 PM
To: user@cassandra.apache.org
Subject: Re: Ring management and load balance

One problem is if the heaviest node is next to a node that's is
lighter than average, instead of heavier.  Then if the new node takes
extra from the heaviest, say 75% instead of just 1/2, and then we take
1/2 of the heaviest's neighbor and put it on the heaviest, you made
that lighter-than-average node even lighter.

Could you move 1/2, 1/4, etc. only until you get to a node lighter
than average?  Probably.  But I'm not sure if it's a big enough win to
justify the the complexity.

Probably a better solution would be a tool where you tell it "I want
to add N nodes to my cluster, analyzes the load factors and tell me
what tokens to add them with, and what additional moves to make to get
me within M% of equal loads, with the minimum amount of data
movement."

-Jonathan

On Thu, Mar 25, 2010 at 1:52 PM, Jeremy Dunck <jdunck@gmail.com> wrote:
> On Thu, Mar 25, 2010 at 1:26 PM, Jonathan Ellis <jbellis@gmail.com> wrote:
>> Pretty much everything assumes that there is a 1:1 correspondence
>> between IP and Token.  It's probably in the ballpark of "one month to
>> code, two to get the bugs out."  Gossip is one of the trickier parts
>> of our code base, and this would be all over that.  The actual storage
>> system changes would be simpler I think.
>
> What if adding a node shifted down-ring tokens less and less?  If
> adding node N+1, it shifts the first N/2^x, the second N/2^2x, the
> third N/2^3x, etc, so that a fixed number of nodes are shifted, but
> the bump is smoothed out?  Tokens stay 1:1.
>
> I'm talking out of my league here -- haven't actually run a cluster
> yet -- so probably a dumb idea.  :-)
>



Mime
View raw message