hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Demai Ni <nid...@gmail.com>
Subject Re: Setting up NxN replication
Date Fri, 08 Nov 2013 21:20:15 GMT
hi, Ishan,

can you please elaborate a bit about your use case?

for your example of 3 cluster, I think it should be set up as two M-S for
the same table:family
first one: A = Master, B = Slave
2nd one: B = slave, C = Master.
There is no need to setup the direct relationship between A and C


On Fri, Nov 8, 2013 at 1:14 PM, Ishan Chhabra <ichhabra@rocketfuel.com>wrote:

> I want to setup NxN replication i.e. N clusters each replicating to each
> other. N is expected to be around 10.
> On doing some research, I realize it is possible after HBASE-7709 fix, but
> it would lead to much more data flowing in the system. eg.
> Lets say we have 3 clusters: A,B and C.
> A new write to A will go to B and then C, and also go to C directly via the
> direct path. This leads to unnecessary network usage and writes to WAL of
> B, that should be avoided. Now imagine this with 10 clusters, it won’t
> scale.
> One option is to create a minimum spanning tree joining all the clusters
> and make nodes replicate to their immediate peers in a master-master
> fashion. This is much better than NxN mesh, but still has extra network and
> WAL usage. It also suffers from a failure scenarios where the a single
> cluster going down will pause replication to clusters downstream.
> What I really want is that the ReplicationSource should only forward
> WALEdits with cluster-id same as the local cluster-id. This seems like a
> straight forward patch to put in.
> Any thoughts on the suggested approach or alternatives?
> --
> *Ishan Chhabra *| Rocket Scientist | RocketFuel Inc.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message