Mailing-List: contact common-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Message-Id: <1465327964.2107285.630780665.10A36D81@webmail.messagingengine.com>
From: Colin McCabe <cmccabe@apache.org>
To: common-dev@hadoop.apache.org
MIME-Version: 1.0
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset="utf-8"
Subject: Re: Why there are so many revert operations on trunk?
Date: Tue, 07 Jun 2016 12:32:44 -0700
In-Reply-To: <CACRbFyhxnnGCz3vdf5RcX-sbsEFKVQepR-n0utDnY4_cmQzwEA@mail.gmail.com>
References: <1465223008024.45875@hortonworks.com>
 <CA+4052nvqDw-1JC6HtkS5R1phHXzS1MwTxL2-KMJaFC+DJhr2w@mail.gmail.com>
 <1465231154205.38002@hortonworks.com>
 <CAGB5D2Z1m3BLJ9EPvWne_9pvN8MCSBS3E-2MpQKL89GZDOMjcQ@mail.gmail.com>
 <846EF988-50E9-48F3-8F33-AEA01A8C35A5@apache.org>
 <1465311433317.96400@hortonworks.com>
 <CAMs9kVj1CXwG1i_kvBsC0EgfjFqOsS2E=C1-nD8RnDQWC_mtNw@mail.gmail.com>
 <CACRbFyhxnnGCz3vdf5RcX-sbsEFKVQepR-n0utDnY4_cmQzwEA@mail.gmail.com>
archived-at: Tue, 07 Jun 2016 19:32:49 -0000

One anti-pattern that keeps coming up over and over again is people
trying to do big and complex features without feature branches.  This
happened with HDFS truncate as well.  This inevitably leads to
controversy because people see very big and invasive patches zooming
past and get alarmed.  Devops people see complicated new code going
directly into production branches and freak out.  Developers see new
APIs getting added without much discussion and start pushing back.

This all gets worse when it's combined with the lack of an up-to-date
design doc.  This leads to people asking the same questions over and
over, since there's no central place they can go to do background
reading about the new feature.  It also tends to lead to arguments,
since the design decisions that were made seem to come out of nowhere,
rather than being justified by careful consideration of alternatives.

When patches get committed directly to production branches, the
discussion also gets mixed up with release management discussions, which
are often contentious as well.  Release management discussions can go on
for a while at the best of times-- when you combine this with all the
above, it just becomes really messy.

My advice is, if 3 or more people ask you for a feature branch, just
create a feature branch already!  It makes things a lot simpler and less
controversial.  It decouples the feature development discussion from the
release management discussion.  It gives developers the ability to look
at the feature as a whole, rather than feeling like they have to
understand each commit in isolation.  Devops people feel like they can
relax because the decision about what stable branches to backport to
will be made after they have a chance to clearly understand the new
feature, rather than before.  The branch merge email functions as a kind
of overview of the feature which makes people more comfortable with it.

Feature branches actually speed up development for medium or large-sized
features.  You can have branch committers who are not full committers,
but can commit to the branch.  You can rewrite your API at any time
without worrying about compatibility.  You can rewrite the history of
the branch however you like to explore new approaches.  You don't need
to backport each of your patches to N stable branches.

Some people seem scared by the need for a merge vote to merge a feature
branch.  But if your feature is big, you are going to get scrutiny
anyway.  Committing stuff directly to trunk or branch-2 is a not a "get
out of jail free" card that bypasses community review.  If anything,
community review will probably be longer and more contentious because of
the reasons above.  People get frustrated when commits to production
branches continue even while big questions about the feature still
remain unanswered.

We already have rules constraining the use of the -1 vote.  Like any
other discussion, -1 votes need to be "constructive"-- that is, they
need to clearly spell out the way forward, rather than just saying no.=20
In this particular case, the concerns we had were about the way the
feature was being developed, and the API.

I think that the discussion that is going on for HDFS async right now is
healthy, and will lead to a better feature.  We have people from
downstream projects such as HBase commenting on the kind of APIs they
would find useful.  We have discussions about the impact on the RPC
system, and the branches that it makes sense to target.  And now that we
have a feature branch, we have a lot more freedom to experiment.

best,
Colin


On Tue, Jun 7, 2016, at 11:02, larry mccay wrote:
> -1 needs not be a taken as a derogatory statement being a number should
> actually make it less emotional.
> It is dangerous to a community to become oversensitive to it.
>=20
> I generally see language such as "I am -1 on this until this particular
> thing is fixed" or that it violates some common pattern or precedence set
> in the project. This is perfectly reasonable language and there is no
> reason to make the reviewer provide an alternative.
>=20
> So, I am giving my -1 to any proposal for rule changes on -1 votes. :)
>=20
>=20
> On Tue, Jun 7, 2016 at 1:15 PM, Ravi Prakash <ravihadoop@gmail.com>
> wrote:
>=20
> > +1 on being more respectful. We seem to be having a lot of distasteful
> > discussions recently. If we fight each other, we are only helping our
> > competitors win (and trust me, its out there).
> >
> > I would also respectfully request people not to throw -1s around. I have
> > faced this a few times and its really frustrating. Every one has opinio=
ns
> > and some times different people can't fathom why someone else thinks the
> > way they do. I am pretty sure none of us is acting with malicious inten=
t,
> > so perhaps a little more tolerance, faith and trust will help all of us
> > improve Hadoop and the ecosystem much faster. That's not to say that
> > sometimes -1s are not warranted, but we should look to it as an extreme
> > measure. Unfortunately there is very little disincentive right now to v=
ote
> > -1 . Maybe we should modify the rules..... if you vote -1 , you have to
> > come up with an alternative implementation? (perhaps limit the amount of
> > time you have to the amount already spent in producing the patch that y=
ou
> > are against)?
> >
> > Just my 2 cents
> > Ravi
> >
> >
> > On Tue, Jun 7, 2016 at 7:54 AM, Junping Du <jdu@hortonworks.com> wrote:
> >
> > > - We need to at the least force a reset of expectations w.r.t how tru=
nk
> > > and small / medium / incompatible changes there are treated. We should
> > hold
> > > off making a release off trunk before this gets fully discussed in the
> > > community and we all reach a consensus.
> > >
> > > +1. We should hold off any release work off trunk before we reach a
> > > consensus. Or more and more developing work/features could be affected
> > just
> > > like Larry mentioned.
> > >
> > >
> > > - Reverts (or revert and move to a feature-branch) shouldn=E2=80=99t =
have been
> > > unequivocally done without dropping a note / informing everyone /
> > building
> > > consensus.
> > >
> > > Agree. To revert commits from other committers, I think we need to: 1)
> > > provide technical evidence/reason that is solid as rack, like: break
> > > functionality, tests, API compatibility, or significantly offend code
> > > convention, etc. 2) Making consensus with related contributors/commit=
ters
> > > based on these technical reasons/evidences. Unfortunately, I didn't s=
ee
> > we
> > > ever do either thing in this case.
> > >
> > >
> > > - Freaking out on -1=E2=80=99s and reverts - we as a community need t=
o be less
> > > stigmatic about -1s / reverts.
> > >
> > > +1. As a community, I believe we all prefer to work in a more friendly
> > > environment. In many cases, -1 without solid reason will frustrate pe=
ople
> > > who are doing contributions. I think we should restraint our -1 unles=
s it
> > > is really necessary.
> > >
> > >
> > >
> > > Thanks,
> > >
> > >
> > > Junping
> > >
> > >
> > > ________________________________
> > > From: Vinod Kumar Vavilapalli <vinodkv@apache.org>
> > > Sent: Monday, June 06, 2016 9:36 PM
> > > To: Andrew Wang
> > > Cc: Junping Du; Aaron T. Myers; common-dev@hadoop.apache.org;
> > > hdfs-dev@hadoop.apache.org; mapreduce-dev@hadoop.apache.org;
> > > yarn-dev@hadoop.apache.org
> > > Subject: Re: Why there are so many revert operations on trunk?
> > >
> > > Folks,
> > >
> > > It is truly disappointing how we are escalating situations that can be
> > > resolved through basic communication.
> > >
> > > Things that shouldn=E2=80=99t have happened
> > > - After a few objections were raised, commits should have simply stop=
ped
> > > before restarting again but only after consensus
> > > - Reverts (or revert and move to a feature-branch) shouldn=E2=80=99t =
have been
> > > unequivocally done without dropping a note / informing everyone /
> > building
> > > consensus. And no, not even a release-manager gets this free pass. No=
t on
> > > branch-2, not on trunk, not anywhere.
> > > - Freaking out on -1=E2=80=99s and reverts - we as a community need t=
o be less
> > > stigmatic about -1s / reverts.
> > >
> > > Trunk releases:
> > > This is the other important bit about huge difference of expectations
> > > between the two sides w.r.t trunk and branching. Till now, we=E2=80=
=99ve never
> > made
> > > releases out of trunk. So in-progress features that people deemed to =
not
> > > need a feature branch could go into trunk without much trouble. Given
> > that
> > > we are now making releases off trunk, I can see (a) the RM saying "no,
> > > don=E2=80=99t put in-progress stuff and (b) the contributors saying =
=E2=80=9Cno we don=E2=80=99t
> > > want the overhead of a branch=E2=80=9D. I=E2=80=99ve raised related t=
opics (but only
> > > focusing on incompatible changes) before -
> > > http://markmail.org/message/m6x73t6srlchywsn - but we never decided
> > > anything.
> > >
> > > We need to at the least force a reset of expectations w.r.t how trunk=
 and
> > > small / medium / incompatible changes there are treated. We should ho=
ld
> > off
> > > making a release off trunk before this gets fully discussed in the
> > > community and we all reach a consensus.
> > >
> > > * Without a user API, there's no way for people to use it, so not much
> > > advantage to having it in a release
> > >
> > > Since the code is separate and probably won't break any existing code=
, I
> > > won't -1 if you want to include this in a release without a user API,=
 but
> > > again, I question the utility of including code that can't be used.
> > >
> > > Clearly, there are two sides to this argument. One side claims the
> > absence
> > > of user-facing public / stable APIs, and that for all purposes this is
> > > dead-code for everyone other than the few early adopters who want to
> > > experiment with it. The other argument is to not put this code before=
 a
> > > user API. Again, I=E2=80=99d discuss with fellow community members be=
fore making
> > > what the other side perceives as unacceptable moves.
> > >
> > > From 2.8.0 perspective, it shouldn=E2=80=99t have landed there in the=
 first place
> > > - I have been pushing for a release for a while with help only from a=
 few
> > > members of the community. But if you say that it has no material impa=
ct
> > on
> > > the user story, having a by-default switched-off feature that *doesn=
=E2=80=99t*
> > > destabilize the core release, I=E2=80=99d be willing to let it pass.
> > >
> > > +Vinod
> > >
> >

---------------------------------------------------------------------
To unsubscribe, e-mail: common-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-dev-help@hadoop.apache.org