Mailing-List: contact issues-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Date: Mon, 9 May 2011 19:39:03 +0000 (UTC)
From: "Andrew Purtell (JIRA)" <jira@apache.org>
To: issues@hbase.apache.org
Message-ID: 
 <389876958.33148.1304969943183.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HBASE-2357) Coprocessors: Add read-only region
 replicas (slaves) for availability and fast region recovery
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HBASE-2357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13030870#comment-13030870 ] 

Andrew Purtell commented on HBASE-2357:
---------------------------------------

@Jason No, no design doc yet. I mean to do one when I can get a suitable block of time for this. 

ZAB is not necessary for basic read replicas that sync "eventually", basic MySQL-like master-slave. That would be the first step of course since most would only need that. Initial thoughts on this is a region slave can get notice from the region owner via zk that a log has rolled and process the new edits from there. Slaves will be under different memory pressure for their mix of regions than the owner, is the only significant detail to work through I think. So for this possibly shadow/temporary flush file storage for slaves that are managing shadow memstores, while sharing the permanent store files with the owner. Also need some zk-mediated coordination around splitting and compaction. Preferably the owner can do splits and compactions leaving the shared store files alone to the last possible moment, then do a change notification via zk and a HDFS rename. And, when all slaves have stopped sharing old storefiles, then garbage collection.

ZAB would be for a next step, getting cliques to all see and agree upon edits coming in, in effect master-master-master replication. This is blue sky stuff. Regions would have higher availability than single region server hosting, yet all clients would have a consistent view of the data contained therein at any moment. However a region would need be deployed to 2N+1 regionservers, where N is the number of expected concurrent node failures, or it would not be writable as long as lacking quorum.

> Coprocessors: Add read-only region replicas (slaves) for availability and fast region recovery
> ----------------------------------------------------------------------------------------------
>
>                 Key: HBASE-2357
>                 URL: https://issues.apache.org/jira/browse/HBASE-2357
>             Project: HBase
>          Issue Type: Sub-task
>          Components: master, regionserver
>            Reporter: Todd Lipcon
>            Assignee: Andrew Purtell
>
> I dont plan on working on this in the short term, but the idea is to extend region ownership to have two modes. Each region has one primary region server and N slave region servers. The slaves would follow the master (probably by streaming the relevant HLog entries directly from it) and be able to serve stale reads. The benefit is twofold: (a) provides the ability to spread read load, (b) enables very fast region failover/rebalance since the memstore is already nearly up to date on the slave RS.

--
This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira