hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Enis Söztutar <e...@apache.org>
Subject [DISCUSSION] Update on HBASE-10070 / Merge into trunk
Date Thu, 22 May 2014 00:08:34 GMT
Hi,

We would like to give an update on the status of HBASE-10070 work, and open
up discussion for how we can do further development.

We seem to be at a point where we have the core functionality of the
region replica, as described in HBASE-10070 working. As pointed out
under the section "Development Phases" in the design doc posted on the
jira HBASE-10070, this work was divided into two broad phases. The first
phase introduces region replicas concept, the new consistency model, and
corresponding RPC implementations. All of the issues for Phase 1 can be
found under [3]. Phase 2 is still in the works, and contains the proposed
changes listed under [4].

With all the issues committed in HBASE-10070 branch in svn, we think that
the "phase-1" is complete. The user documentation on HBASE-10513 gives an
accurate picture of what has been done in phase-1 and what the impact of
using this feature is, APIs etc. We have added
a couple of IT tests as part of this work and we have tested the work
we did in "phase-1" of the project quite extensively in Hortonworks'
infrastructure.

In summary, with the code in branch, you can create tables with region
replicas, do gets / multi gets and scans using TIMELINE consistency with
high availability. Region replicas periodically scan the files of the
primary and pick up flushed / committed files. The RPC paths / assignment,
balancing etc are pretty stable. However some more performance analysis and
tuning is needed. More information can be found in [1] and [2].


As a reminder, the development has been happening in the branch -
hbase-10070 (https://github.com/apache/hbase/tree/hbase-10070). We have
been pretty diligent about more than one committer's +1 on the branch
commits and for almost all the subtasks in HBASE-10070 there is more than
one +1 except for test fix issues or more trivial issues, where there maybe
 only one +1.  Enis/Nicolas/Sergey/Devaraj/Nick are the main contributors
of code in the phase-1 and many patches have been reviewed already by
people outside
this group (mainly Stack, Jimmy)

For Phase 2, we think that we can deliver on the proposed design
incrementally over the next couple of months. However, we think that it
might be ok to merge the changes from phase 1 first, then do a
commit-as-you-go approach for remaining items. Therefore, we would like to
propose  to merge the branch to trunk, and continue the work over there.
This might also result in more reviews.

Alternatively, we can continue the work in the branch, and do the merge at
the end of Phase 2, but that will make the review process a bit more
tricky, which is why we would like the merge sooner.

What do you think? Comments, concerns?

[1]
https://issues.apache.org/jira/secure/attachment/12644237/hbase-10513_v1.patch
[2]
http://www.slideshare.net/enissoz/hbase-high-availability-for-reads-with-time
[3] https://issues.apache.org/jira/browse/HBASE-10070
[4] https://issues.apache.org/jira/browse/HBASE-11183

Thanks,

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message