Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
MIME-Version: 1.0
From: Andrew Purtell <apurtell@apache.org>
Date: Mon, 7 Sep 2015 22:43:26 -0700
Message-ID: 
 <CA+RK=_C-vv4ytDnM6bwuZB6aYndt8aYNUZcQLEwSejPQWvt+xQ@mail.gmail.com>
Subject: Considering a completely different administrative approach for
 Replication v2
To: "dev@hbase.apache.org" <dev@hbase.apache.org>
Content-Type: multipart/alternative; boundary=089e01537f16a88b66051f35da5e

--089e01537f16a88b66051f35da5e
Content-Type: text/plain; charset=UTF-8

I opened an umbrella for Replication v2 as HBASE-14379. At the moment it
envisions the administration of cross-DC replication relationships and data
access as the same as today. However, we do have an opportunity to reboot
with a completely different approach. I thought it worth bringing up for
discussion.

We could in theory reboot around timeline consistent region replicas. If
you squint, region replicas have a similar theory of operation as cross-DC
replication. What if we redefine administration and data access for
Replication v2 as sets of region replica placements that can cross data
center boundaries, with the client able to distinguish local locations from
remote locations, and then choose based on policy? So if, for example, you
may have three data centers, then instead of setting up three
point-to-point replication peering relationships like today, you'd simply
create a table that has a region replica placement policy in its schema
with (logical) locations spanning all three data centers. Behind the
scenes, each data center would have HBASE-10070 style primary-secondary
relationships, and additionally:
1. the primary region will run something like today's replication source
for each secondary location that is in a remote DC;
2. a primary region anywhere may receive change streams from remote DCs
like today's replication sinks.

On the client side we have some prior work in this regard: CSBT, and Ted
Malaska's HBase.MCC. I mention CSBT but I don't think we want its
partitioning or reliance on Zookeeper. HBase.MCC is more of a starting
point.

I'm not saying we should do this, only that we could do this. There are
pros and cons. In some ways defining point-to-point replication
relationships is easier for admins and users, e.g. the topology is built
and managed explicitly. In some ways merging replicas and cross-DC
replication is easier, e.g. it removes APIs, necessary tooling, cognitive
load (cross-DC replication is no longer 'special').

-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

--089e01537f16a88b66051f35da5e--