hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Francis Liu <tof...@apache.org>
Subject Re: DISCUSS: Merge new Assignment Manager (AMv2), HBASE-14614
Date Thu, 25 May 2017 17:53:20 GMT
 > The list of items for 2.0 are here [1]. I think you are talking about
> something that is NOT in this doc. If so, yeah, we are way behind for 2.0.

I see good to know, I was mainly asking for split meta and favored nodes both of which are
in the doc. FN has most of the stuff in and split meta I can start rebasing on AMv2. 
> Perhaps we could add a few place-holders for your incoming in a 2.0 so it
> could come in at a later time, in a 2.1 (or 3.0 if it was close behind)?
This is good to know as well. 

Thanks,Francis

    On Wednesday, May 24, 2017 5:24 PM, Stack <stack@duboce.net> wrote:
 

 On Wed, May 24, 2017 at 1:56 PM, Francis Liu <toffer@apache.org> wrote:

> I'd be up for taking this for a spin. As I need to get split meta rebased
> on top of this anyway.


Great.



> Also I'd generally like to know more about AMv2 (transition from zkless
> etc).



See the linked doc. If you need more, call it out and I'll fill it in.



> BTW once 2.0 is branched does that mean we can't merge in any big features
> anymore?
> Thanks,Francis
>
>
The list of items for 2.0 are here [1]. I think you are talking about
something that is NOT in this doc. If so, yeah, we are way behind for 2.0.
Perhaps we could add a few place-holders for your incoming in a 2.0 so it
could come in at a later time, in a 2.1 (or 3.0 if it was close behind)?

Thanks,
S

1.
https://docs.google.com/document/d/1WCsVlnHjJeKUcl7wHwqb4z9iEu_ktczrlKHK8N4SZzs/edit#heading=h.v21r9nz8g01j



>
>
>
>    On Wednesday, May 24, 2017 8:37 AM, Stack <stack@duboce.net> wrote:
>
>
>  On Wed, May 24, 2017 at 8:11 AM, Sean Busbey <busbey@apache.org> wrote:
>
> > Presuming merge happens, looking at the TODOs I see 1 called out
> > expressly as a blocker and 2 called out as minor. Is it safe to assume
> > the remainder have been evaluated and found some where in between, or
> > will we need a triaging step once they're in JIRA to determine number
> > of blockers before branch-2 would be ready for release?
> >
> > The former.
>
> The item called out as a BLOCKER I have not been unable to reproduce in a
> few weeks now. I should probably knock down its priority given this fact
> (and that should it happen -- we are talking about dropping Procedures
> because of Procedure WAL Corruption -- it will be for hbck to cover until
> source of corruption is figured).
>
> I do not know of any other BLOCKERs, not at this point. We want hbase2
> rolling upgradeable from hbase1 but that I see as a separate task, one we
> might as a community want to punt on, and besides, AMv2, as I see it, would
> be a prerequisite.
>
> The others are tasks of some substance, worthy of call out, but for the
> most part improvement and finishing.
>
> No harm in me filing JIRAs for the non-nebulous of [1]. Let me work on this
> today.
>
> Thanks Sean,
> St.Ack
>
> 1.
> https://docs.google.com/document/d/1eVKa7FHdeoJ1-
> 9o8yZcOTAQbv0u0bblBlCCzVSIn69g/edit#heading=h.xp9zndoycwj
>
>
>
> > On Wed, May 24, 2017 at 2:22 AM, Stack <stack@duboce.net> wrote:
> > > I would like to discuss merging branch HBASE-14614 to master.
> > >
> > > The new AssignmentManager on HBASE-14614 branch promises new levels of
> > > scaling, insight, resilience and assignment performance. A skeleton dev
> > > overview is available here [1].
> > >
> > > It has been in development a good while now. Currently it is working at
> > > least as well as our current assign. Cluster testing has me into a new
> > > realm of scale where the failures are for reasons other than AM; e.g.
> > > massive WALs that take tens of minutes to split holding up restore post
> > > crash (TODO).
> > >
> > > The AMv2 patch is large, unfortunately. Most of the bulk comes of
> > generated
> > > protobuf changes but the patch is also large because the commit could
> not
> > > deliver an half-baked machine.
> > >
> > > The patch is in need of review. Independent confirmation that it is
> > > basically working would be sweet.
> > >
> > > There are outstanding items to complete but they can come in
> post-merge.
> > > Critical is reenabling a set of tests disabled because they depend on
> > > workings no longer present or they depended on a broken behavior in
> AMv1
> > > (Disabled tests and the list of TODOs are in [1]).
> > >
> > > Currently I am working on polish making it pass the outstanding failing
> > > unit tests, fixing white space, and findbugs but it is ready for
> commit.
> > > I've put up an RB patch that is absent protobufs so the bulk is less
> > > intimidating [3].
> > >
> > > This feature is the last hurdle to our branching for hbase2.
> > >
> > > Thanks,
> > > St.Ack
> > >
> > > P.S. This patch is mostly the work of Matteo Bertozzi (inception, bulk
> of
> > > implementation). Others have contributed including Stephen Yuan Jiang.
> > > Umesh Agashe and Appy also have contributed fixes that got added to the
> > > HBASE-14614 branch.
> > >
> > > 1.
> > > https://docs.google.com/document/d/1eVKa7FHdeoJ1-
> > 9o8yZcOTAQbv0u0bblBlCCzVSIn69g/edit#
> > > 2.
> > > https://docs.google.com/document/d/1QLXlVERKt5EMbx_
> > EL3Y2u0j64FN-_TrVoM5WWxIXh6o/edit#heading=h.df9krsl9k16
> > > 3. https://reviews.apache.org/r/55117/
> >
>
>
>
>


   
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message