hbase-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jean-Daniel Cryans <jdcry...@apache.org>
Subject Re: HBase Cyclic Replication Issue: some data are missing in the replication for intensive write
Date Mon, 23 Apr 2012 17:57:52 GMT
> I will try your suggestion today with a master-slave replication enabled from Cluster
A -> Cluster B.

Please do.

> Last Friday, I tried to limit the variability/the moving part of the replication components.
I reduced the size of Cluster B to have only 1 regionserver and having Cluster A to replicate
data from one region only without region splitting (therefore I have 1-to-1 region replication
setup). During the benchmark, I moved the region between different regionservers in Cluster
A (note there are still 3 regionservers in Cluster A). I ran this test for 5 times and no
data were lost. Does it mean something? My feeling is there are some glitches/corner cases
that have not been covered in the cyclic replication (or hbase replication in general). Note
that, this happens only when the load is high.

And have you looked at the logs? Any obvious exceptions coming up?
Replication uses the normal HBase client to insert the data on the
other cluster and this is what handles regions moving around.

>
> By the way, why do we need to have a zookeeper not handled by hbase for the replication
to work (it is described in the hbase documentation)?

It says you *should* do it, not you *need* to do it :)

But basically replication is zk-heavy and getting a better
understanding of it starts with handling it yourself.

J-D

Mime
View raw message