hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Purtell (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-2357) Coprocessors: Add read-only region replicas (slaves) for availability and fast region recovery
Date Mon, 09 May 2011 19:39:03 GMT

    [ https://issues.apache.org/jira/browse/HBASE-2357?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13030870#comment-13030870

Andrew Purtell commented on HBASE-2357:

@Jason No, no design doc yet. I mean to do one when I can get a suitable block of time for

ZAB is not necessary for basic read replicas that sync "eventually", basic MySQL-like master-slave.
That would be the first step of course since most would only need that. Initial thoughts on
this is a region slave can get notice from the region owner via zk that a log has rolled and
process the new edits from there. Slaves will be under different memory pressure for their
mix of regions than the owner, is the only significant detail to work through I think. So
for this possibly shadow/temporary flush file storage for slaves that are managing shadow
memstores, while sharing the permanent store files with the owner. Also need some zk-mediated
coordination around splitting and compaction. Preferably the owner can do splits and compactions
leaving the shared store files alone to the last possible moment, then do a change notification
via zk and a HDFS rename. And, when all slaves have stopped sharing old storefiles, then garbage

ZAB would be for a next step, getting cliques to all see and agree upon edits coming in, in
effect master-master-master replication. This is blue sky stuff. Regions would have higher
availability than single region server hosting, yet all clients would have a consistent view
of the data contained therein at any moment. However a region would need be deployed to 2N+1
regionservers, where N is the number of expected concurrent node failures, or it would not
be writable as long as lacking quorum.

> Coprocessors: Add read-only region replicas (slaves) for availability and fast region
> ----------------------------------------------------------------------------------------------
>                 Key: HBASE-2357
>                 URL: https://issues.apache.org/jira/browse/HBASE-2357
>             Project: HBase
>          Issue Type: Sub-task
>          Components: master, regionserver
>            Reporter: Todd Lipcon
>            Assignee: Andrew Purtell
> I dont plan on working on this in the short term, but the idea is to extend region ownership
to have two modes. Each region has one primary region server and N slave region servers. The
slaves would follow the master (probably by streaming the relevant HLog entries directly from
it) and be able to serve stale reads. The benefit is twofold: (a) provides the ability to
spread read load, (b) enables very fast region failover/rebalance since the memstore is already
nearly up to date on the slave RS.

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message