hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Enis Söztutar <enis....@gmail.com>
Subject Re: [DISCUSS] Criteria for including MOB feature backport in branch-1
Date Thu, 03 Mar 2016 23:49:20 GMT
Regardless of the backport, did we do the same regression analysis for the
master branch merge at the time of the merge? Like make sure that the
latencies and stability of non-mob is affected or not. Sorry, I was not
following the merge vote closely.

The reason I am asking is that although we can allow some flexibility in
master, we still want to keep master releasable and prevent a 1-year effort
to re-stabilize master when we want to release 2.0. If we have design /
stability / impact questions, when and how they will be addressed for a 2.0
release? Let's say that we would like to release 2.0 by summer time as
Matteo was quoting, how do we make sure that the feature as is, is stable
and supportable?

Enis

On Thu, Mar 3, 2016 at 3:36 PM, Andrew Purtell <apurtell@apache.org> wrote:

> Just to be clear, I did not actually oppose a backport of MOB. Although - I
> have to say am sympathetic to Elliott's position that there's a project
> management reason not to. My insistence here is "merely" for this change in
> particular a backport to the branch we are making production releases from
> demands a credible data driven assessment on stability, functionality, and
> performance - both avoidance of regression and affirmative indication of
> the alleged benefit.
>
> On Thu, Mar 3, 2016 at 3:16 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > In the middle of writing response to this thread, I saw subsequent
> comments
> > from Andrew, Jon and Elliott.
> >
> > In light of opposition from the community w.r.t. backporting MOB
> feature, I
> > think it suffices to say that this wouldn't be done now.
> >
> > Thanks everyone for chiming in.
> >
> > On Thu, Mar 3, 2016 at 2:42 PM, Elliott Clark <eclark@apache.org> wrote:
> >
> > > I just don't see a why we would back port. We're going to release a 2.0
> > > when things are ready. It will be a major feature release. MOB is a
> major
> > > feature. Without compelling reason backporting to branch-1 seems like
> an
> > > end run around what sem ver is supposed to mean (not the api
> guarantees,
> > > the actual meaning).
> > >
> > > Backporting major features is a bad habit that the Hadoop community
> seems
> > > to have. We shouldn't follow their lead.
> > >
> > > On Thu, Mar 3, 2016 at 1:01 PM, Stack <stack@duboce.net> wrote:
> > >
> > > > On Thu, Mar 3, 2016 at 9:02 AM, Ted Yu <yuzhihong@gmail.com> wrote:
> > > >
> > > > > Hi,
> > > > > As requested by Sean Busbey, I am starting a new thread w.r.t.
> > > > backporting
> > > > > MOB feature to branch-1.
> > > > >
> > > > This is to solicit discussion on the criteria for including MOB
> feature
> > > > > backport in branch-1.
> > > > >
> > > > > I can think of the following:
> > > > > 1. whether there is customer interest
> > > > >
> > > > > There is.
> > > > > See Jonathan's response here:
> > > > http://search-hadoop.com/m/YGbbDSqxD1PYXK62
> > > > >
> > > > >
> > > > You should probably mention that a note requesting if there was
> > interest
> > > > got no response here on the public lists. This would seem to imply no
> > > > interest by the community?
> > > >
> > > > Jon's note is pregnant but opaque on the details of these 'supported'
> > > > deploys. He is in a bit of an awkward spot unable to share detail on
> > > > someone else's deploy.
> > > >
> > > > Would be good to see more interest than Jon's note as evidence of
> > > interest
> > > > I'd say.
> > > >
> > > > You know of any? Even if they could be described in outline and
> > hopefully
> > > > more than one instance and that MOB makes sense for this deploy. And
> > they
> > > > need it now in a branch-1?
> > > >
> > > > Which version of hbase are we talking of backporting too? 1.3? 1.4?
> > > >
> > > >
> > > >
> > > > > 2. whether unit test stability can be maintained in branch-1
> > > > >
> > > > > Inclusion of the backport should keep unit tests (both existing and
> > > new)
> > > > in
> > > > > branch-1 green.
> > > > > Preliminary test runs showed that MOB / snapshot related tests
> > > > consistently
> > > > > pass.
> > > > >
> > > > >
> > > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-15370?focusedCommentId=15176094&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15176094
> > > >
> > > >
> > > > Hopefully we are aiming for more than just 'test stability'. Some
> large
> > > > community users have placed big bets on branch-1 being stable at
> scale;
> > > > this is the stability we should be precious of -- not just 'test
> > > > stability'.
> > > >
> > > > Besides, I see failures in your listing. MOB is pervasive. Why is it
> > not
> > > > responsible for the mentioned test failures? (And "... passes on my
> > local
> > > > machine... " doesn't cut it; builds.apache.org is our public CI
> where
> > > > community comes together on state of branch and master, not some devs
> > > > laptop).
> > > >
> > > > Branch-1 builds have been generally stable after large investment
> > fixing,
> > > > tuning and pruning tests. Master branch had been rendered stable but
> > > seems
> > > > to be rotting again though it is the most important of our builds
> given
> > > it
> > > > can catch the bad stuff before the commits make it in. During the
> > > campaign
> > > > to stabilize Master, MOB tests failed often. I see MOB failures in
> > master
> > > > patch build from time to time still (I've been lax of late but our
> > > flakies
> > > > list contains a few mentionds, see HBASE-15012). They go unaddressed
> > > > (though, to be fair, Jingcheng, the original author, addressed a few
> > > > failures early on when asked). Just this morning I note:
> > > >
> > > >
> > >
> >
> https://builds.apache.org/view/H-L/view/HBase/job/PreCommit-HBASE-Build/818/testReport/junit/org.apache.hadoop.hbase.mob.mapreduce/TestMobSweepMapper/org_apache_hadoop_hbase_mob_mapreduce_TestMobSweepMapper/
> > > >
> > > >
> > > > (Suggestion: MOB seems to be better in master branch of late --
> Matteo
> > > and
> > > > Heng work I believe (or just my disabling of broke tests) -- so a
> > review
> > > > and report of master and patch builds looking for MOB failures and
> > fixing
> > > > any found would help address the above concern. Another way to build
> > > > confidence in a patched branch-1 would be doing the branch suggested
> up
> > > in
> > > > the backport issue and then running builds on apache for a period. Or
> > > > better, copy what was done by Jon et al. to build confidence; run
> > > > long-running ITBLLs on a cluster w/ MOB set with obnoxious configs
> and
> > > post
> > > > evidence it all holds up).
> > > >
> > > >
> > > > > Comments are welcome
> > > > >
> > > > > <https://issues.apache.org/jira/browse/HBASE-15370#>
> > > > >
> > > >
> > > >
> > > > Is MOB an 'owned' feature. The MOB original author is mostly absent
> > (for
> > > > instance, no participation in these conversations and no general
> > > follow-up
> > > > on test failures). As has been said before many times, features need
> to
> > > be
> > > > owned. Writing the code is just one part of owning a feature.
> Features
> > > that
> > > > are not owned become a burden on the community to maintain. We have
> > > enough
> > > > of this latter type of feature already.
> > > >
> > > > (Jingcheng did show up in the last day though with
> > HBASE-15381"Implement
> > > a
> > > > distributed MOB compaction by procedure" which looks like a pretty
> > > > important issue and begs the question, is MOB finished? And if not,
> > > > shouldn't we wait till its done before we backport?)
> > > >
> > > > Finally, MOB backport should be done by someone who is familiar with
> > > MOB. I
> > > > see no evidence of your expertise in MOB other than an odd review nor
> > > even
> > > > that you've run it in other than a unit test mode. Not to mention
> that
> > > the
> > > > way you go about the backport, dumping an 800k blob of unattributed
> > code
> > > > into the issue for review in HBASE-15370, does not bode well for our
> > > > continued stability.
> > > >
> > > > St.Ack
> > > >
> > >
> >
>
>
>
> --
> Best regards,
>
>    - Andy
>
> Problems worthy of attack prove their worth by hitting back. - Piet Hein
> (via Tom White)
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message