Mailing-List: contact yarn-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-dev@hadoop.apache.org
Received-SPF: pass (nike.apache.org: message received from 54.76.25.247 which
 is an MX secondary for yarn-dev@hadoop.apache.org)
MIME-Version: 1.0
In-Reply-To: 
 <CADbBEnvnLqWB3FThHJughyP_xdsswOeovmMQPC-GSGiQhBsBVQ@mail.gmail.com>
References: 
 <CALwhT95ZnrOa=8ex36gnz=ac-XBWEDHY6Xq9nQtiV+qmj6e1cQ@mail.gmail.com>
	<79459119-DEA9-4D77-8BC6-86EBFF983739@altiscale.com>
	<CAHz+ZFchV32j==Jg7w5kheh7Bkx+EFXUQb0ocT-Evp+NztUZNA@mail.gmail.com>
	<CADbBEnvnLqWB3FThHJughyP_xdsswOeovmMQPC-GSGiQhBsBVQ@mail.gmail.com>
Date: Tue, 28 Apr 2015 17:28:23 -0700
Message-ID: 
 <CALwhT94JKXfjOhWv52uHC_3ZSzguMgLH8qo5JvruVW3boQAOsA@mail.gmail.com>
Subject: Re: [DISCUSS] Developing features in branches
From: Karthik Kambatla <kasha@cloudera.com>
To: "yarn-dev@hadoop.apache.org" <yarn-dev@hadoop.apache.org>
Content-Type: multipart/alternative; boundary=089e0122eaea94c9700514d20ef0

--089e0122eaea94c9700514d20ef0
Content-Type: text/plain; charset=UTF-8

On Tue, Apr 28, 2015 at 3:58 PM, Sangjin Lee <sjlee0@gmail.com> wrote:

> Having worked in both modes, I can see the pros and cons. The branch
> definitely helps with speed of development as you don't necessarily have to
> make it so that each JIRA has to work standalone and clear the higher bar
> and so on.
>
> That said, in a way we're deferring the cost of cleaning things up towards
> the end of the branch. For example, we don't get the same treatment of the
> hadoop jenkins in a branch development. It's left up to the group or the
> individuals to make sure to run test-patch.sh to ensure tech debt does not
> accumulate.
>

Allen's comment above and his recent work on test-patch.sh mostly
alleviates this problem.

My take is the cost of cleaning things up at the end is small if all
features are on branches. It is particularly hard if only some features use
branches.


>
> I would agree that big feature developments can benefit from branch
> development, but there should be enough discipline around it so we don't
> miss on the quality during the development. My 2 cents.
>
> On Tue, Apr 28, 2015 at 11:10 AM, Robert Kanter <rkanter@cloudera.com>
> wrote:
>
> > +1
> >
> > A good example of both sides of this are the ATSv1 and ATSv2.  v1 was not
> > done in a feature branch, and was included piecemeal in Hadoop releases
> in
> > various states.  In the end, it's design doesn't scale and there's a few
> > other problems that aren't going to be fixed; and we're already working
> on
> > ATSv2, which is in a feature branch this time.  This means we can pull it
> > in if/when it's ready.
> >
> > - Robert
> >
> > On Tue, Apr 28, 2015 at 9:59 AM, Allen Wittenauer <aw@altiscale.com>
> > wrote:
> >
> > >
> > >         For major features, yes, +1.  Labels, for example, should have
> > > really been done in a branch and I was shocked that it wasn't.
> > >
> > >         Anything that helps get quality back up.  The fact that our
> > > "stable" branch isn't is a very very bad thing.  There is way too much
> > > Pokemon happening in Hadoop:
> > >
> > >         "This feature doesn't actually work for anything but this one
> use
> > > case."
> > >
> > >         "Oh, we're not finished with it yet.  You haven't seen it's
> final
> > > form."
> > >
> > >         2 releases later....
> > >
> > >         "Oh, we stopped working on it/not a priority."
> > >
> > >         *grumble*
> > >
> > >         It's worth mentioning that we have a successful confirmation
> that
> > > the new test-patch.sh is properly testing on feature branches if patch
> > > files are named appropriately and branches are named in a sane manner
> > > (hint: avoid periods and use only one -).  See HDFS-7859 for the
> details
> > of
> > > a run last night on the HDFS-7285 feature branch.
> > >
> > > On Apr 28, 2015, at 9:26 AM, Karthik Kambatla <kasha@cloudera.com>
> > wrote:
> > >
> > > > Hi Yarn devs,
> > > >
> > > > I wanted to hear your thoughts on moving to a model where we develop
> > ALL
> > > > features in branches. I see the following pros/cons of this approach:
> > > >
> > > > Pros:
> > > >
> > > >   1. A feature gets included in a release only after it is in a
> usable
> > > >   state - security etc. included. e.g. SharedCache has been included
> in
> > > 2.7
> > > >   partially. We are yet to have MR use it or enable security.
> > > >   2. Since it is not in a branch a release might be cut from, feature
> > > >   development could move faster without having to make sure the
> branch
> > > is in
> > > >   releasable-state between commits. e.g. Working on YARN-149 was hard
> > > mostly
> > > >   because we wanted the state between commits to be pristine.
> > > >   3. Provide contributors the opportunity to review and commit code
> as
> > > >   branch-committers.
> > > >
> > > > Con: The cost of merging the branch back to trunk and branch-2. It
> > might
> > > be
> > > > painful to resolve conflicts during every rebase. However, if
> > > > trunk/branch-2 have primarily bug fixes, there won't be as many
> > > conflicts.
> > > >
> > > > From what I hear, the HDFS team has followed this model somewhat
> > > > successfully. In YARN land, recent work on reservations was done in a
> > > > branch and I think it went well.
> > > >
> > > > This is one of those cases where switching completely to a
> > > > features-in-branches model is lot more convenient than trying it out
> > for
> > > > one-off features.
> > > >
> > > > Thanks
> > > > Karthik
> > > >
> > > > --
> > > > Karthik Kambatla
> > > > Software Engineer, Cloudera Inc.
> > > > --------------------------------------------
> > >
> > >
> >
>


-- 
Karthik Kambatla
Software Engineer, Cloudera Inc.
--------------------------------------------
http://five.sentenc.es

--089e0122eaea94c9700514d20ef0--