hbase-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jonathan Hsieh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HBASE-10070) HBase read high-availability using eventually consistent region replicas
Date Tue, 03 Dec 2013 01:58:35 GMT

    [ https://issues.apache.org/jira/browse/HBASE-10070?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13837234#comment-13837234
] 

Jonathan Hsieh commented on HBASE-10070:
----------------------------------------

High level comparison after first read: These are  are quite complimentary on goals and initial
focuses.

* enis focuses on stale read availability for phase 1, then reduced recovery time for phase
2.  Jon focuses on reduced recovery time first, and then punts on stale read availability
for a phase 2.
* enis provides 3 options for log recovery, jon focuses on what enis calls the "wal tailing"
approach.

The requirement that I'm most concerned about *recovering consistent reads as quickly as possible*.
 I find the memstore / wal tail appealing most because we are essentially focusing the most
on this current area of weakness.

At the moment I've done some experiments on the wal tailing approach to see how much overhead
it causes on the nn. I don't think we need to have a wal-per-region world to make it feasible
-- we only need to make sure that our shadow region are more a more like shadow regionservers
 -- only have one or two of the replicas read the logs and make them the preferred places
for the shadows.  This basically acts as a constraint for the selection of where secondaries
are assigned.   

[~enis] in your doc does "group-based assignment" mean assigning multiple regions on a single
transaction?   or is this hdfs affinity groups?  (if it is, then I agree, we need hdfs affinity
groups for this to be efficient).  However, I don't think we get into a situation where all
RS's must read all other RS's logs -- we only need to have the shadows RS's to read the primary
RS's log.


> HBase read high-availability using eventually consistent region replicas
> ------------------------------------------------------------------------
>
>                 Key: HBASE-10070
>                 URL: https://issues.apache.org/jira/browse/HBASE-10070
>             Project: HBase
>          Issue Type: New Feature
>            Reporter: Enis Soztutar
>            Assignee: Enis Soztutar
>         Attachments: HighAvailabilityDesignforreadsApachedoc.pdf
>
>
> In the present HBase architecture, it is hard, probably impossible, to satisfy constraints
like 99th percentile of the reads will be served under 10 ms. One of the major factors that
affects this is the MTTR for regions. There are three phases in the MTTR process - detection,
assignment, and recovery. Of these, the detection is usually the longest and is presently
in the order of 20-30 seconds. During this time, the clients would not be able to read the
region data.
> However, some clients will be better served if regions will be available for reads during
recovery for doing eventually consistent reads. This will help with satisfying low latency
guarantees for some class of applications which can work with stale reads.
> For improving read availability, we propose a replicated read-only region serving design,
also referred as secondary regions, or region shadows. Extending current model of a region
being opened for reads and writes in a single region server, the region will be also opened
for reading in region servers. The region server which hosts the region for reads and writes
(as in current case) will be declared as PRIMARY, while 0 or more region servers might be
hosting the region as SECONDARY. There may be more than one secondary (replica count >
2).
> Will attach a design doc shortly which contains most of the details and some thoughts
about development approaches. Reviews are more than welcome. 
> We also have a proof of concept patch, which includes the master and regions server side
of changes. Client side changes will be coming soon as well. 



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message