Mailing-List: contact dev-help@hbase.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@hbase.apache.org
MIME-Version: 1.0
Sender: saint.ack@gmail.com
In-Reply-To: 
 <CALte62z6ZgbOdbHUJHzydq1SvDaMEy9F_KZoROL4+CKJUu1OPw@mail.gmail.com>
References: 
 <CALte62z6ZgbOdbHUJHzydq1SvDaMEy9F_KZoROL4+CKJUu1OPw@mail.gmail.com>
Date: Thu, 3 Mar 2016 13:01:08 -0800
Message-ID: 
 <CADcMMgGbbH+vxDzWFe=OuHHR2mg8=obXZz=Q=8uN6FhXP6p=xg@mail.gmail.com>
Subject: Re: [DISCUSS] Criteria for including MOB feature backport in branch-1
From: Stack <stack@duboce.net>
To: HBase Dev List <dev@hbase.apache.org>
Content-Type: multipart/alternative; boundary=001a113ceea62da4c5052d2b4cb0

--001a113ceea62da4c5052d2b4cb0
Content-Type: text/plain; charset=UTF-8

On Thu, Mar 3, 2016 at 9:02 AM, Ted Yu <yuzhihong@gmail.com> wrote:

> Hi,
> As requested by Sean Busbey, I am starting a new thread w.r.t. backporting
> MOB feature to branch-1.
>
This is to solicit discussion on the criteria for including MOB feature
> backport in branch-1.
>
> I can think of the following:
> 1. whether there is customer interest
>
> There is.
> See Jonathan's response here: http://search-hadoop.com/m/YGbbDSqxD1PYXK62
>
>
You should probably mention that a note requesting if there was interest
got no response here on the public lists. This would seem to imply no
interest by the community?

Jon's note is pregnant but opaque on the details of these 'supported'
deploys. He is in a bit of an awkward spot unable to share detail on
someone else's deploy.

Would be good to see more interest than Jon's note as evidence of interest
I'd say.

You know of any? Even if they could be described in outline and hopefully
more than one instance and that MOB makes sense for this deploy. And they
need it now in a branch-1?

Which version of hbase are we talking of backporting too? 1.3? 1.4?


> 2. whether unit test stability can be maintained in branch-1
>
> Inclusion of the backport should keep unit tests (both existing and new) in
> branch-1 green.
> Preliminary test runs showed that MOB / snapshot related tests consistently
> pass.
>
>
> https://issues.apache.org/jira/browse/HBASE-15370?focusedCommentId=15176094&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15176094


Hopefully we are aiming for more than just 'test stability'. Some large
community users have placed big bets on branch-1 being stable at scale;
this is the stability we should be precious of -- not just 'test stability'.

Besides, I see failures in your listing. MOB is pervasive. Why is it not
responsible for the mentioned test failures? (And "... passes on my local
machine... " doesn't cut it; builds.apache.org is our public CI where
community comes together on state of branch and master, not some devs
laptop).

Branch-1 builds have been generally stable after large investment fixing,
tuning and pruning tests. Master branch had been rendered stable but seems
to be rotting again though it is the most important of our builds given it
can catch the bad stuff before the commits make it in. During the campaign
to stabilize Master, MOB tests failed often. I see MOB failures in master
patch build from time to time still (I've been lax of late but our flakies
list contains a few mentionds, see HBASE-15012). They go unaddressed
(though, to be fair, Jingcheng, the original author, addressed a few
failures early on when asked). Just this morning I note:
https://builds.apache.org/view/H-L/view/HBase/job/PreCommit-HBASE-Build/818/testReport/junit/org.apache.hadoop.hbase.mob.mapreduce/TestMobSweepMapper/org_apache_hadoop_hbase_mob_mapreduce_TestMobSweepMapper/


(Suggestion: MOB seems to be better in master branch of late -- Matteo and
Heng work I believe (or just my disabling of broke tests) -- so a review
and report of master and patch builds looking for MOB failures and fixing
any found would help address the above concern. Another way to build
confidence in a patched branch-1 would be doing the branch suggested up in
the backport issue and then running builds on apache for a period. Or
better, copy what was done by Jon et al. to build confidence; run
long-running ITBLLs on a cluster w/ MOB set with obnoxious configs and post
evidence it all holds up).


> Comments are welcome
>
> <https://issues.apache.org/jira/browse/HBASE-15370#>
>


Is MOB an 'owned' feature. The MOB original author is mostly absent (for
instance, no participation in these conversations and no general follow-up
on test failures). As has been said before many times, features need to be
owned. Writing the code is just one part of owning a feature. Features that
are not owned become a burden on the community to maintain. We have enough
of this latter type of feature already.

(Jingcheng did show up in the last day though with HBASE-15381"Implement a
distributed MOB compaction by procedure" which looks like a pretty
important issue and begs the question, is MOB finished? And if not,
shouldn't we wait till its done before we backport?)

Finally, MOB backport should be done by someone who is familiar with MOB. I
see no evidence of your expertise in MOB other than an odd review nor even
that you've run it in other than a unit test mode. Not to mention that the
way you go about the backport, dumping an 800k blob of unattributed code
into the issue for review in HBASE-15370, does not bode well for our
continued stability.

St.Ack

--001a113ceea62da4c5052d2b4cb0--