hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: DISCUSS: Merge new Assignment Manager (AMv2), HBASE-14614
Date Wed, 24 May 2017 22:49:13 GMT
How can we help test this? Build a dev branch and deploy to test clusters
running long lived integration tests?

On Wed, May 24, 2017 at 12:22 AM, Stack <stack@duboce.net> wrote:

> I would like to discuss merging branch HBASE-14614 to master.
>
> The new AssignmentManager on HBASE-14614 branch promises new levels of
> scaling, insight, resilience and assignment performance. A skeleton dev
> overview is available here [1].
>
> It has been in development a good while now. Currently it is working at
> least as well as our current assign. Cluster testing has me into a new
> realm of scale where the failures are for reasons other than AM; e.g.
> massive WALs that take tens of minutes to split holding up restore post
> crash (TODO).
>
> The AMv2 patch is large, unfortunately. Most of the bulk comes of generated
> protobuf changes but the patch is also large because the commit could not
> deliver an half-baked machine.
>
> The patch is in need of review. Independent confirmation that it is
> basically working would be sweet.
>
> There are outstanding items to complete but they can come in post-merge.
> Critical is reenabling a set of tests disabled because they depend on
> workings no longer present or they depended on a broken behavior in AMv1
> (Disabled tests and the list of TODOs are in [1]).
>
> Currently I am working on polish making it pass the outstanding failing
> unit tests, fixing white space, and findbugs but it is ready for commit.
> I've put up an RB patch that is absent protobufs so the bulk is less
> intimidating [3].
>
> This feature is the last hurdle to our branching for hbase2.
>
> Thanks,
> St.Ack
>
> P.S. This patch is mostly the work of Matteo Bertozzi (inception, bulk of
> implementation). Others have contributed including Stephen Yuan Jiang.
> Umesh Agashe and Appy also have contributed fixes that got added to the
> HBASE-14614 branch.
>
> 1.
> https://docs.google.com/document/d/1eVKa7FHdeoJ1-
> 9o8yZcOTAQbv0u0bblBlCCzVSIn69g/edit#
> 2.
> https://docs.google.com/document/d/1QLXlVERKt5EMbx_
> EL3Y2u0j64FN-_TrVoM5WWxIXh6o/edit#heading=h.df9krsl9k16
> 3. https://reviews.apache.org/r/55117/
>



-- 
Best regards,

   - Andy

If you are given a choice, you believe you have acted freely. - Raymond
Teller (via Peter Watts)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message