hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrew Purtell <apurt...@apache.org>
Subject Re: [DISCUSS] Criteria for including MOB feature backport in branch-1
Date Thu, 03 Mar 2016 23:36:21 GMT
Just to be clear, I did not actually oppose a backport of MOB. Although - I
have to say am sympathetic to Elliott's position that there's a project
management reason not to. My insistence here is "merely" for this change in
particular a backport to the branch we are making production releases from
demands a credible data driven assessment on stability, functionality, and
performance - both avoidance of regression and affirmative indication of
the alleged benefit.

On Thu, Mar 3, 2016 at 3:16 PM, Ted Yu <yuzhihong@gmail.com> wrote:

> In the middle of writing response to this thread, I saw subsequent comments
> from Andrew, Jon and Elliott.
>
> In light of opposition from the community w.r.t. backporting MOB feature, I
> think it suffices to say that this wouldn't be done now.
>
> Thanks everyone for chiming in.
>
> On Thu, Mar 3, 2016 at 2:42 PM, Elliott Clark <eclark@apache.org> wrote:
>
> > I just don't see a why we would back port. We're going to release a 2.0
> > when things are ready. It will be a major feature release. MOB is a major
> > feature. Without compelling reason backporting to branch-1 seems like an
> > end run around what sem ver is supposed to mean (not the api guarantees,
> > the actual meaning).
> >
> > Backporting major features is a bad habit that the Hadoop community seems
> > to have. We shouldn't follow their lead.
> >
> > On Thu, Mar 3, 2016 at 1:01 PM, Stack <stack@duboce.net> wrote:
> >
> > > On Thu, Mar 3, 2016 at 9:02 AM, Ted Yu <yuzhihong@gmail.com> wrote:
> > >
> > > > Hi,
> > > > As requested by Sean Busbey, I am starting a new thread w.r.t.
> > > backporting
> > > > MOB feature to branch-1.
> > > >
> > > This is to solicit discussion on the criteria for including MOB feature
> > > > backport in branch-1.
> > > >
> > > > I can think of the following:
> > > > 1. whether there is customer interest
> > > >
> > > > There is.
> > > > See Jonathan's response here:
> > > http://search-hadoop.com/m/YGbbDSqxD1PYXK62
> > > >
> > > >
> > > You should probably mention that a note requesting if there was
> interest
> > > got no response here on the public lists. This would seem to imply no
> > > interest by the community?
> > >
> > > Jon's note is pregnant but opaque on the details of these 'supported'
> > > deploys. He is in a bit of an awkward spot unable to share detail on
> > > someone else's deploy.
> > >
> > > Would be good to see more interest than Jon's note as evidence of
> > interest
> > > I'd say.
> > >
> > > You know of any? Even if they could be described in outline and
> hopefully
> > > more than one instance and that MOB makes sense for this deploy. And
> they
> > > need it now in a branch-1?
> > >
> > > Which version of hbase are we talking of backporting too? 1.3? 1.4?
> > >
> > >
> > >
> > > > 2. whether unit test stability can be maintained in branch-1
> > > >
> > > > Inclusion of the backport should keep unit tests (both existing and
> > new)
> > > in
> > > > branch-1 green.
> > > > Preliminary test runs showed that MOB / snapshot related tests
> > > consistently
> > > > pass.
> > > >
> > > >
> > > >
> > >
> >
> https://issues.apache.org/jira/browse/HBASE-15370?focusedCommentId=15176094&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15176094
> > >
> > >
> > > Hopefully we are aiming for more than just 'test stability'. Some large
> > > community users have placed big bets on branch-1 being stable at scale;
> > > this is the stability we should be precious of -- not just 'test
> > > stability'.
> > >
> > > Besides, I see failures in your listing. MOB is pervasive. Why is it
> not
> > > responsible for the mentioned test failures? (And "... passes on my
> local
> > > machine... " doesn't cut it; builds.apache.org is our public CI where
> > > community comes together on state of branch and master, not some devs
> > > laptop).
> > >
> > > Branch-1 builds have been generally stable after large investment
> fixing,
> > > tuning and pruning tests. Master branch had been rendered stable but
> > seems
> > > to be rotting again though it is the most important of our builds given
> > it
> > > can catch the bad stuff before the commits make it in. During the
> > campaign
> > > to stabilize Master, MOB tests failed often. I see MOB failures in
> master
> > > patch build from time to time still (I've been lax of late but our
> > flakies
> > > list contains a few mentionds, see HBASE-15012). They go unaddressed
> > > (though, to be fair, Jingcheng, the original author, addressed a few
> > > failures early on when asked). Just this morning I note:
> > >
> > >
> >
> https://builds.apache.org/view/H-L/view/HBase/job/PreCommit-HBASE-Build/818/testReport/junit/org.apache.hadoop.hbase.mob.mapreduce/TestMobSweepMapper/org_apache_hadoop_hbase_mob_mapreduce_TestMobSweepMapper/
> > >
> > >
> > > (Suggestion: MOB seems to be better in master branch of late -- Matteo
> > and
> > > Heng work I believe (or just my disabling of broke tests) -- so a
> review
> > > and report of master and patch builds looking for MOB failures and
> fixing
> > > any found would help address the above concern. Another way to build
> > > confidence in a patched branch-1 would be doing the branch suggested up
> > in
> > > the backport issue and then running builds on apache for a period. Or
> > > better, copy what was done by Jon et al. to build confidence; run
> > > long-running ITBLLs on a cluster w/ MOB set with obnoxious configs and
> > post
> > > evidence it all holds up).
> > >
> > >
> > > > Comments are welcome
> > > >
> > > > <https://issues.apache.org/jira/browse/HBASE-15370#>
> > > >
> > >
> > >
> > > Is MOB an 'owned' feature. The MOB original author is mostly absent
> (for
> > > instance, no participation in these conversations and no general
> > follow-up
> > > on test failures). As has been said before many times, features need to
> > be
> > > owned. Writing the code is just one part of owning a feature. Features
> > that
> > > are not owned become a burden on the community to maintain. We have
> > enough
> > > of this latter type of feature already.
> > >
> > > (Jingcheng did show up in the last day though with
> HBASE-15381"Implement
> > a
> > > distributed MOB compaction by procedure" which looks like a pretty
> > > important issue and begs the question, is MOB finished? And if not,
> > > shouldn't we wait till its done before we backport?)
> > >
> > > Finally, MOB backport should be done by someone who is familiar with
> > MOB. I
> > > see no evidence of your expertise in MOB other than an odd review nor
> > even
> > > that you've run it in other than a unit test mode. Not to mention that
> > the
> > > way you go about the backport, dumping an 800k blob of unattributed
> code
> > > into the issue for review in HBASE-15370, does not bode well for our
> > > continued stability.
> > >
> > > St.Ack
> > >
> >
>



-- 
Best regards,

   - Andy

Problems worthy of attack prove their worth by hitting back. - Piet Hein
(via Tom White)

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message