hbase-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: MOB in branch-1? (Re: [RESULT][VOTE] Merge branch hbase-11339 HBase MOB to trunk)
Date Thu, 03 Mar 2016 03:57:08 GMT
Jon:
Mind taking a look at this attachment for list of commits (in case I missed
any):

https://issues.apache.org/jira/secure/attachment/12791047/mob-cmmits.txt


Thanks

On Wed, Mar 2, 2016 at 7:26 PM, Jonathan Hsieh <jon@cloudera.com> wrote:

> Most of the patches should have some mention of "mob" in their titles.
> Another sure sign is to look for commits attributed to jingcheng du -- he
> contributed the majority of the patches.  I did a few mostly adding flags
> for testing and that bulk load related bug fix.
>
> I'll review the list of patches as part of any branch merge proposal to
> make sure all the necessary ones are present.
>
> On more minor thing -- instead of the longish hbase-11339-branch-1 name I
> proposed -- naming the branch the name umbrella jira of the backport
> (hbase-15370) would make it even easier to track provenance of the branch.
>
> Jon.
>
> On Wed, Mar 2, 2016 at 6:44 PM, Devaraj Das <ddas@hortonworks.com> wrote:
>
> > Sounds good to me, Jonathan. Although I suspect that at the end this will
> > be close to what Ted Yu has in the backport patch but this is a more
> > structured / reliable way of doing it. When you say this - "backport the
> > patches that came after the merge to trunk....", is there a reliable way
> to
> > identify all those. In the backport jira, there is a mention of some
> > additional jiras after the original merge - https://goo.gl/6stN7Y - we
> > need to make sure if these are the only ones relevant.
> > ________________________________________
> > From: Jonathan Hsieh <jon@cloudera.com>
> > Sent: Wednesday, March 02, 2016 2:03 PM
> > To: dev@hbase.apache.org
> > Subject: Re: MOB in branch-1? (Re: [RESULT][VOTE] Merge branch
> hbase-11339
> > HBase MOB to trunk)
> >
> > I'll offer what I think would be another reasonable and likely less
> > destabilizing approach:  essentially treat it as a branch.  We could
> start
> > from the existing hbase-11339 branch, call it hbase-11339-branch-1 and do
> > the work to merge it into branch-1 (instead of trunk).  To merge into
> > branch-1 it would require the 3 +1's just like any other branch merge.
> >
> > Mob initially started off as a mega patch and was broken down and
> committed
> > in reviewable pieces to branch hbase-11339.  Most of this was actually
> when
> > 1.0.0-SNAPSHOT was trunk so many of the older commits may actually jive
> > better with branch-1. Towards the end we did periodic merge commits with
> > trunk as we shook out issues integration testing found, and as branch
> merge
> > reviews uncovered other short comings.  These merge patches actually
> showed
> > how we modified the mob code to account for the later changes in trunk.
> In
> > this particular case we coud take hbase-11339-branch-1,  backport the
> > patches that came after the merge to trunk with the normal single +1
> review
> > process on the branch, and then propose a branch merge with branch-1.
> When
> > all pieces are ready and we had stable unit tests (admittedly that was a
> > one shortcoming when I sheparded hbase-11339) we do 3 +1s to merge to
> > branch-1 (and possibly branch-1.3).
> >
> > This has the nice benefit compared to andrew's suggesting of not
> > potentially blocking branch-1 or having a half complete mob feature in
> > branch-1 if hbase-11339-branch-1 isn't ready.
> >
> > Jon.
> >
> >
> >
> >
> > On Wed, Mar 2, 2016 at 1:31 PM, Andrew Purtell <andrew.purtell@gmail.com
> >
> > wrote:
> >
> > > On the question of change provenance, if I can make a suggestion: How I
> > > would approach the backport of a feature developed in trunk over
> multiple
> > > commits and JIRAs is I would look through git history, find each
> related
> > > commit, and apply each commit in sequence to branch-1, fixing things up
> > as
> > > needed each time. Then the resulting working branch can be pushed up to
> > aid
> > > review of the aggregate result by the MOB developers and the larger
> > > community. Ted, if you approached the backport in this way, then
> pushing
> > up
> > > your working branch will aid review.
> > >
> > > > On Mar 2, 2016, at 1:17 PM, Andrew Purtell <andrew.purtell@gmail.com
> >
> > > wrote:
> > > >
> > > > I concur with Stack's concerns with a developer not involved in
> > building
> > > the MOB feature attempting a backport in general, and furthermore
> without
> > > detailed provenance of all of the incorporated changes. I'm also
> > concerned
> > > about how well MOB works in real production given people use branch-1
> in
> > > production and not trunk.
> > > >
> > > > As for Ted's issue, I'm not going to let him commit it (I will veto)
> > > without clear proof that it works and doesn't hurt perf of non MOB
> cases
> > by
> > > way of reproducible benchmarks - benchmarks I can personally reproduce
> > > afterward (I'm not volunteering to do Ted's necessary legwork) - done
> > with
> > > branch-1 with the proposed patch applied. We are not there yet though,
> > > barely, into review, but this will be coming up very soon.
> > > >
> > > >>> On Mar 2, 2016, at 1:03 PM, Stack <stack@duboce.net> wrote:
> > > >>>
> > > >>> On Wed, Mar 2, 2016 at 11:26 AM, Devaraj Das <ddas@hortonworks.com
> >
> > > wrote:
> > > >>>
> > > >>> Stack, as things stand, Ted Yu has a patch that is a backport
of
> MOB.
> > > He
> > > >>> told me offline he has taken a look at the jiras that went in
in
> the
> > > master
> > > >>> that is to do with MOB, and got them in the backport.
> > > >>
> > > >>
> > > >>
> > > >> On the issue, a near 800k blob is dumped which is described as "...a
> > > >> backport of MOB... " but w/o attribution of provenance nor any other
> > > >> description of what it contains. Only when this is pointed-out in
> the
> > > issue
> > > >> do we get a short listing of supposed JIRAs included with no
> > > justification
> > > >> of what is covered, what is included/excluded, and what machinations
> > > were
> > > >> done to make it fit branch-1 (Yet you offline were given this
> info?).
> > > >>
> > > >> Such poor practice only makes me more intent on my objection.
> > > >>
> > > >>
> > > >>
> > > >>> Now, to your point, I agree that someone familiar with MOB code
> > should
> > > do
> > > >>> the backport but the question is, is that dev available to do
it
> now?
> > > The
> > > >>> next best thing is to get Ted Yu's patch reviewed by someone
> familiar
> > > with
> > > >>> the feature. I really hope that we can get cycles from the original
> > MOB
> > > >>> devs on this.
> > > >>
> > > >>
> > > >> MOB is unsupported? If so, lets for sure not backport it.
> > > >>
> > > >>
> > > >> Agree that we shouldn't be adding flaky tests. The question is if
> the
> > > >>> failures on master to do with MOB are really MOB related or
> something
> > > else
> > > >>> (for argument's sake, let's say, procv2)..
> > > >> Sounds like we need to spend a bit of time digging in on the flakey
> > > tests
> > > >> then. Branch-1 is pretty stable now after a bunch of expended effort
> > by
> > > a
> > > >> few folks. Would be a pity taking a step back.
> > > >>
> > > >> St.Ack
> > > >>
> > > >>
> > > >>
> > > >>> On the point about the DISCUSS thread, yeah, it's fine to do it
if
> > > folks
> > > >>> feel it's the right way to go.
> > > >>> ________________________________________
> > > >>> From: saint.ack@gmail.com <saint.ack@gmail.com> on behalf
of
> Stack <
> > > >>> stack@duboce.net>
> > > >>> Sent: Wednesday, March 02, 2016 10:55 AM
> > > >>> To: HBase Dev List
> > > >>> Subject: Re: MOB in branch-1? (Re: [RESULT][VOTE] Merge branch
> > > hbase-11339
> > > >>> HBase MOB to trunk)
> > > >>>
> > > >>> (Doing another resend...)
> > > >>>
> > > >>> I have objection to you, Ted Yu, doing it. MOB has spread all
about
> > > master
> > > >>> branch. Backport should be done by someone who knows MOB.
> > > >>>
> > > >>> Higher up in this thread, Sean asks: "Can we get a [DISCUSS]
> flagged
> > > thread
> > > >>> to see what, if anything, folks
> > > >>> would like to see gate inclusion in branch-1?"  Shouldn't we do
> this
> > > before
> > > >>> we 'create a backport...'.
> > > >>>
> > > >>> For me, there should be no new flakies in branch-1. Branch-1 builds
> > > are a
> > > >>> hard-won stable(-ish). Looking at master build, MOB seems quiet
> > lately
> > > but
> > > >>> going by HBASE-15012, our flakies umbrella issue, I see notes
that
> > > >>> TestMobExportSnapshot has failed a few times (that is probably
> > because
> > > the
> > > >>> test it derives from is flakey) and
> > > TestMobRestoreFlushSnapshotFromClient
> > > >>> gets a mention.
> > > >>>
> > > >>> St.Ack
> > > >>>
> > > >>>
> > > >>>
> > > >>>> On Tue, Mar 1, 2016 at 5:27 PM, Stack <stack@duboce.net>
wrote:
> > > >>>>
> > > >>>> (Doing a resend of below... it doesn't seem to have gone through)
> > > >>>>
> > > >>>>> On Mon, Feb 29, 2016 at 8:20 PM, Ted Yu <yuzhihong@gmail.com>
> > wrote:
> > > >>>>>
> > > >>>>> If there is no objection, I will create a backport JIRA
tomorrow
> > and
> > > >>>>> attach patch.
> > > >>>> I have objection to you doing it. MOB has spread all about
master
> > > branch.
> > > >>>> Backport should be done by someone who knows MOB.
> > > >>>>
> > > >>>> Higher up in this thread, Sean asks: "Can we get a [DISCUSS]
> flagged
> > > >>>> thread to see what, if anything, folks
> > > >>>> would like to see gate inclusion in branch-1?"  Shouldn't
we do
> this
> > > >>>> before we 'create a backport...'.
> > > >>>>
> > > >>>> For me, there should be no new flakies in branch-1. Branch-1
> builds
> > > are a
> > > >>>> hard-won stable(-ish). Looking at master build, MOB seems
quiet
> > lately
> > > >>> but
> > > >>>> going by HBASE-15012, our flakies umbrella issue, I see notes
that
> > > >>>> TestMobExportSnapshot has failed a few times (that is probably
> > because
> > > >>> the
> > > >>>> test it derives from is flakey) and
> > > TestMobRestoreFlushSnapshotFromClient
> > > >>>> gets a mention.
> > > >>>>
> > > >>>> St.Ack
> > > >>>>
> > > >>>>
> > > >>>>
> > > >>>>>> On Tue, Mar 1, 2016 at 9:32 AM, Stack <stack@duboce.net>
wrote:
> > > >>>>>>
> > > >>>>>> On Mon, Feb 29, 2016 at 8:20 PM, Ted Yu <yuzhihong@gmail.com>
> > > wrote:
> > > >>>>>>
> > > >>>>>> If there is no objection, I will create a backport
JIRA tomorrow
> > and
> > > >>>>>> attach patch.
> > > >>>>> I have objection to you doing it. MOB has spread all about
master
> > > >>> branch.
> > > >>>>> Backport should be done by someone who knows MOB.
> > > >>>>>
> > > >>>>> Higher up in this thread, Sean asks: "Can we get a [DISCUSS]
> > flagged
> > > >>>>> thread to see what, if anything, folks
> > > >>>>> would like to see gate inclusion in branch-1?"  Shouldn't
we do
> > this
> > > >>>>> before we 'create a backport...'.
> > > >>>>>
> > > >>>>> For me, there should be no new flakies in branch-1. Branch-1
> builds
> > > are
> > > >>> a
> > > >>>>> hard-won stable(-ish). Looking at master build, MOB seems
quiet
> > > lately
> > > >>> but
> > > >>>>> going by HBASE-15012, our flakies umbrella issue, I see
notes
> that
> > > >>>>> TestMobExportSnapshot has failed a few times (that is
probably
> > > because
> > > >>> the
> > > >>>>> test it derives from is flakey) and
> > > >>> TestMobRestoreFlushSnapshotFromClient
> > > >>>>> gets a mention.
> > > >>>>>
> > > >>>>> St.Ack
> > > >>>
> > >
> >
> >
> >
> > --
> > // Jonathan Hsieh (shay)
> > // HBase Tech Lead, Software Engineer, Cloudera
> > // jon@cloudera.com // @jmhsieh
> >
>
>
>
> --
> // Jonathan Hsieh (shay)
> // HBase Tech Lead, Software Engineer, Cloudera
> // jon@cloudera.com // @jmhsieh
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message