hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bikas Saha <bi...@hortonworks.com>
Subject RE: [DISCUSS] Developing features in branches
Date Tue, 28 Apr 2015 23:02:15 GMT
+1

-----Original Message-----
From: Sangjin Lee [mailto:sjlee0@gmail.com] 
Sent: Tuesday, April 28, 2015 3:59 PM
To: yarn-dev@hadoop.apache.org
Subject: Re: [DISCUSS] Developing features in branches

Having worked in both modes, I can see the pros and cons. The branch definitely helps with
speed of development as you don't necessarily have to make it so that each JIRA has to work
standalone and clear the higher bar and so on.

That said, in a way we're deferring the cost of cleaning things up towards the end of the
branch. For example, we don't get the same treatment of the hadoop jenkins in a branch development.
It's left up to the group or the individuals to make sure to run test-patch.sh to ensure tech
debt does not accumulate.

I would agree that big feature developments can benefit from branch development, but there
should be enough discipline around it so we don't miss on the quality during the development.
My 2 cents.

On Tue, Apr 28, 2015 at 11:10 AM, Robert Kanter <rkanter@cloudera.com>
wrote:

> +1
>
> A good example of both sides of this are the ATSv1 and ATSv2.  v1 was 
> not done in a feature branch, and was included piecemeal in Hadoop 
> releases in various states.  In the end, it's design doesn't scale and 
> there's a few other problems that aren't going to be fixed; and we're 
> already working on ATSv2, which is in a feature branch this time.  
> This means we can pull it in if/when it's ready.
>
> - Robert
>
> On Tue, Apr 28, 2015 at 9:59 AM, Allen Wittenauer <aw@altiscale.com>
> wrote:
>
> >
> >         For major features, yes, +1.  Labels, for example, should 
> > have really been done in a branch and I was shocked that it wasn't.
> >
> >         Anything that helps get quality back up.  The fact that our 
> > "stable" branch isn't is a very very bad thing.  There is way too 
> > much Pokemon happening in Hadoop:
> >
> >         "This feature doesn't actually work for anything but this 
> > one use case."
> >
> >         "Oh, we're not finished with it yet.  You haven't seen it's 
> > final form."
> >
> >         2 releases later....
> >
> >         "Oh, we stopped working on it/not a priority."
> >
> >         *grumble*
> >
> >         It's worth mentioning that we have a successful confirmation 
> > that the new test-patch.sh is properly testing on feature branches 
> > if patch files are named appropriately and branches are named in a 
> > sane manner
> > (hint: avoid periods and use only one -).  See HDFS-7859 for the 
> > details
> of
> > a run last night on the HDFS-7285 feature branch.
> >
> > On Apr 28, 2015, at 9:26 AM, Karthik Kambatla <kasha@cloudera.com>
> wrote:
> >
> > > Hi Yarn devs,
> > >
> > > I wanted to hear your thoughts on moving to a model where we 
> > > develop
> ALL
> > > features in branches. I see the following pros/cons of this approach:
> > >
> > > Pros:
> > >
> > >   1. A feature gets included in a release only after it is in a usable
> > >   state - security etc. included. e.g. SharedCache has been 
> > > included in
> > 2.7
> > >   partially. We are yet to have MR use it or enable security.
> > >   2. Since it is not in a branch a release might be cut from, feature
> > >   development could move faster without having to make sure the 
> > > branch
> > is in
> > >   releasable-state between commits. e.g. Working on YARN-149 was 
> > > hard
> > mostly
> > >   because we wanted the state between commits to be pristine.
> > >   3. Provide contributors the opportunity to review and commit code as
> > >   branch-committers.
> > >
> > > Con: The cost of merging the branch back to trunk and branch-2. It
> might
> > be
> > > painful to resolve conflicts during every rebase. However, if
> > > trunk/branch-2 have primarily bug fixes, there won't be as many
> > conflicts.
> > >
> > > From what I hear, the HDFS team has followed this model somewhat 
> > > successfully. In YARN land, recent work on reservations was done 
> > > in a branch and I think it went well.
> > >
> > > This is one of those cases where switching completely to a 
> > > features-in-branches model is lot more convenient than trying it 
> > > out
> for
> > > one-off features.
> > >
> > > Thanks
> > > Karthik
> > >
> > > --
> > > Karthik Kambatla
> > > Software Engineer, Cloudera Inc.
> > > --------------------------------------------
> >
> >
>

Mime
View raw message