hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stack <st...@duboce.net>
Subject Re: DISCUSS: Merge new Assignment Manager (AMv2), HBASE-14614
Date Thu, 25 May 2017 00:20:34 GMT
On Wed, May 24, 2017 at 8:37 AM, Stack <stack@duboce.net> wrote:

> On Wed, May 24, 2017 at 8:11 AM, Sean Busbey <busbey@apache.org> wrote:
>
>> ....
>
>
> No harm in me filing JIRAs for the non-nebulous of [1]. Let me work on
> this today.
>

I filed 9 new subtasks against HBASE-14350 Procedure V2 Phase 2: Assignment
Manager (I also closed almost as many issues). One is critical but a small
amount of work (a 'cleaner' for the archived Master Procedure WAL files).
The others are non-critical and their prioritization reflects that.

I updated the doc. with links to the JIRAs.

Thanks,
St.Ack




> Thanks Sean,
> St.Ack
>
> 1. https://docs.google.com/document/d/1eVKa7FHdeoJ1-
> 9o8yZcOTAQbv0u0bblBlCCzVSIn69g/edit#heading=h.xp9zndoycwj
>
>
>
>> On Wed, May 24, 2017 at 2:22 AM, Stack <stack@duboce.net> wrote:
>> > I would like to discuss merging branch HBASE-14614 to master.
>> >
>> > The new AssignmentManager on HBASE-14614 branch promises new levels of
>> > scaling, insight, resilience and assignment performance. A skeleton dev
>> > overview is available here [1].
>> >
>> > It has been in development a good while now. Currently it is working at
>> > least as well as our current assign. Cluster testing has me into a new
>> > realm of scale where the failures are for reasons other than AM; e.g.
>> > massive WALs that take tens of minutes to split holding up restore post
>> > crash (TODO).
>> >
>> > The AMv2 patch is large, unfortunately. Most of the bulk comes of
>> generated
>> > protobuf changes but the patch is also large because the commit could
>> not
>> > deliver an half-baked machine.
>> >
>> > The patch is in need of review. Independent confirmation that it is
>> > basically working would be sweet.
>> >
>> > There are outstanding items to complete but they can come in post-merge.
>> > Critical is reenabling a set of tests disabled because they depend on
>> > workings no longer present or they depended on a broken behavior in AMv1
>> > (Disabled tests and the list of TODOs are in [1]).
>> >
>> > Currently I am working on polish making it pass the outstanding failing
>> > unit tests, fixing white space, and findbugs but it is ready for commit.
>> > I've put up an RB patch that is absent protobufs so the bulk is less
>> > intimidating [3].
>> >
>> > This feature is the last hurdle to our branching for hbase2.
>> >
>> > Thanks,
>> > St.Ack
>> >
>> > P.S. This patch is mostly the work of Matteo Bertozzi (inception, bulk
>> of
>> > implementation). Others have contributed including Stephen Yuan Jiang.
>> > Umesh Agashe and Appy also have contributed fixes that got added to the
>> > HBASE-14614 branch.
>> >
>> > 1.
>> > https://docs.google.com/document/d/1eVKa7FHdeoJ1-9o8yZcOTAQb
>> v0u0bblBlCCzVSIn69g/edit#
>> > 2.
>> > https://docs.google.com/document/d/1QLXlVERKt5EMbx_EL3Y2u0j6
>> 4FN-_TrVoM5WWxIXh6o/edit#heading=h.df9krsl9k16
>> > 3. https://reviews.apache.org/r/55117/
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message