> (2) is where we get into trouble here no matter which DC we add to.
> (a) if we add to G's DC, X will get all the replicas G has, remaining
> unbalanced
> (b) if we add to the other DC, G will still be hit from all the
> replicas from the other DC
2b: yes
2a: not necessarily. Let's return once more to the original ring:
A: HA, FG
B: AB, HA
C: BC, HA, AB
D: CD, BC
E: DE, CD
F: EF, DE
G: FG, AB, BC, CD, DE, EF
H: GH, EF, FG
Let us suppose that all ranges are equal in size. In this case G's
range is AG. If X boots in G's DC, it should take a token in the
middle of this range, which would be somewhere around D. If X boots
behind D, the ring would be:
A: HA, FG
B: AB, HA
C: BC, HA, AB
D: CD, BC
X: DX, AB, BC, CD
E: XE, CD, DX
F: EF, XE
G: FG, DX, EF
H: GH, EF, FG
In this case X takes approximately half of G's load, since it takes
approximately half of replicas G was responsible for.
IMHO a new node should take half of the most loaded node *in the same
DC*. If it only considers most loaded node, things go wrong, as
booting in the middle of that range in another DC will not balance the
load in any way.
> So ISTM that the only real solution is to do what we say in the
> Operations page, and make sure that nodes on the ring alternate DCs.
Yes, alternating the nodes is certainly the best. However, two DCs
don't always have the same number of nodes. Also, currently
loadbalance is unusable in such environment.
Jaakko
