hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject DISCUSS: Merge new Assignment Manager (AMv2), HBASE-14614
Date Wed, 24 May 2017 07:22:37 GMT
I would like to discuss merging branch HBASE-14614 to master.

The new AssignmentManager on HBASE-14614 branch promises new levels of
scaling, insight, resilience and assignment performance. A skeleton dev
overview is available here [1].

It has been in development a good while now. Currently it is working at
least as well as our current assign. Cluster testing has me into a new
realm of scale where the failures are for reasons other than AM; e.g.
massive WALs that take tens of minutes to split holding up restore post
crash (TODO).

The AMv2 patch is large, unfortunately. Most of the bulk comes of generated
protobuf changes but the patch is also large because the commit could not
deliver an half-baked machine.

The patch is in need of review. Independent confirmation that it is
basically working would be sweet.

There are outstanding items to complete but they can come in post-merge.
Critical is reenabling a set of tests disabled because they depend on
workings no longer present or they depended on a broken behavior in AMv1
(Disabled tests and the list of TODOs are in [1]).

Currently I am working on polish making it pass the outstanding failing
unit tests, fixing white space, and findbugs but it is ready for commit.
I've put up an RB patch that is absent protobufs so the bulk is less
intimidating [3].

This feature is the last hurdle to our branching for hbase2.


P.S. This patch is mostly the work of Matteo Bertozzi (inception, bulk of
implementation). Others have contributed including Stephen Yuan Jiang.
Umesh Agashe and Appy also have contributed fixes that got added to the
HBASE-14614 branch.

3. https://reviews.apache.org/r/55117/

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message