hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: DISCUSS: Merge new Assignment Manager (AMv2), HBASE-14614
Date Tue, 30 May 2017 15:25:00 GMT
Discussion has gone quiet. Let me move to a vote thread.

On Wed, May 24, 2017 at 12:22 AM, Stack <stack@duboce.net> wrote:

> I would like to discuss merging branch HBASE-14614 to master.
> The new AssignmentManager on HBASE-14614 branch promises new levels of
> scaling, insight, resilience and assignment performance. A skeleton dev
> overview is available here [1].
> It has been in development a good while now. Currently it is working at
> least as well as our current assign. Cluster testing has me into a new
> realm of scale where the failures are for reasons other than AM; e.g.
> massive WALs that take tens of minutes to split holding up restore post
> crash (TODO).
> The AMv2 patch is large, unfortunately. Most of the bulk comes of
> generated protobuf changes but the patch is also large because the commit
> could not deliver an half-baked machine.
> The patch is in need of review. Independent confirmation that it is
> basically working would be sweet.
> There are outstanding items to complete but they can come in post-merge.
> Critical is reenabling a set of tests disabled because they depend on
> workings no longer present or they depended on a broken behavior in AMv1
> (Disabled tests and the list of TODOs are in [1]).
> Currently I am working on polish making it pass the outstanding failing
> unit tests, fixing white space, and findbugs but it is ready for commit.
> I've put up an RB patch that is absent protobufs so the bulk is less
> intimidating [3].
> This feature is the last hurdle to our branching for hbase2.
> Thanks,
> St.Ack
> P.S. This patch is mostly the work of Matteo Bertozzi (inception, bulk of
> implementation). Others have contributed including Stephen Yuan Jiang.
> Umesh Agashe and Appy also have contributed fixes that got added to the
> HBASE-14614 branch.
> 1. https://docs.google.com/document/d/1eVKa7FHdeoJ1-
> 9o8yZcOTAQbv0u0bblBlCCzVSIn69g/edit#
> 2. https://docs.google.com/document/d/1QLXlVERKt5EMbx_
> EL3Y2u0j64FN-_TrVoM5WWxIXh6o/edit#heading=h.df9krsl9k16
> 3. https://reviews.apache.org/r/55117/

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message