Mailing-List: contact general-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: general@hadoop.apache.org
Received-SPF: pass (nike.apache.org: domain of todd@cloudera.com designates
 209.85.214.48 as permitted sender)
MIME-Version: 1.0
In-Reply-To: <C9EF6BA2.355E9%ddas@yahoo-inc.com>
References: <BANLkTi=38uUOYGP4zvPfoqPHC3KEKrKnXA@mail.gmail.com>
 <C9EF6BA2.355E9%ddas@yahoo-inc.com>
From: Todd Lipcon <todd@cloudera.com>
Date: Wed, 11 May 2011 00:00:06 -0700
Message-ID: <BANLkTiks+0+wqTtZ7UA_UV_vG8Zqf=uD2Q@mail.gmail.com>
Subject: Re: Release compatibility was Re: [VOTE] Release candidate
 0.20.203.0-rc1
To: general@hadoop.apache.org
Content-Type: multipart/alternative; boundary=00032555903e920d9604a2faa044

--00032555903e920d9604a2faa044
Content-Type: text/plain; charset=ISO-8859-1

On Tue, May 10, 2011 at 10:24 PM, Devaraj Das <ddas@yahoo-inc.com> wrote:

> Just so that everyone is on the same page w.r.t the compatibility between
> 20.2 & 20.203 (don't think this is documented anywhere yet)..
>
> The aim of the team working on Hadoop Security at Yahoo! was to make Hadoop
> *secure*, and with *minimal* disruption to existing apps. I can't think of
> any major user-facing incompatibilities other than the users having to run a
> 'kinit' when they are working with a secure Hadoop cluster (of course the
> admins need to do more work in order to set up a secure cluster). Also,
> security can switched off, and all the other enhancements (job limits, etc.)
> are still available.. As per users/Operations/Solutions at Yahoo!,
> 20.security was one of the smoothest upgrades ever.
>

And I think you guys did a commendable job with this, given the scope of the
project! :)

But there were certainly plenty of bugs introduced along the way that
affected both secure and non-secure, and even now the security-able branches
don't function on any non-Sun JVM.

Again, I think for this particular case, most of the developers agreed on
the risk/reward trade-off, so I didn't want to start a discussion about
security being a good or bad decision to backport on to 0.20.

But, I'd love to know what our framework is for making such decisions in the
future, if we plan to maintain branches with feature backports as part of
Apache. (eg what scope of change requires what type of vote and when)

-Todd


>
> On 5/10/11 2:28 PM, "Todd Lipcon" <todd@cloudera.com> wrote:
>
> On Tue, May 10, 2011 at 12:41 PM, Scott Carey <scott@richrelevance.com
> >wrote:
>
> >
> > As an observer, this is a very important observation.  Sure, the default
> > is that dot releases are bugfix-onl.  But exceptions to these rules are
> > sometimes required and often beneficial to the health of the project.
> > Performance enhancements, minor features, and other items are sometimes
> > very low risk and the barrier to getting them to users earlier should be
> > lower.
> >
>
> I agree whole-heartedly.
>
>
> > These issues are the sort of things that get into non-Apache releases
> > quickly and drive the community away from the Apache release.  Its been
> > well proven through those vehicles that back-porting minor features and
> > improvements from trunk to an old release can be done safely.
>
>
> However, one shouldn't understate the difficulty of agreeing on the
> risk-reward tradeoff here. While risk is mostly technical, reward may vary
> widely based on the userbase or organization.
>
> For example, everyone would agree that security was a very risky feature to
> add to 20, with known backward compatibilities and a lot of fallout. For
> some people (both CDH and YDH), the security features were an absolute
> necessity on a tight timeline, so the risk-reward decision was clear --
> I've
> heard from many users, though, that they saw none of the reward from
> security and wished they hadn't had to endure the resulting changes and
> bugs
> within the 0.20 series.
>
> Another example is the 0.20-append patch series, which is indispensable for
> the HBase community but seen as overly risky by those who do not use HBase.
>
> So, while I'm in favor of "sustaining" release series like 0.20-security in
> theory, I also think we need a clear inclusion criteria for such branches.
> As I said in a previous email, the criteria used to be "low risk compatible
> bug fixes only" with a vote process for any exceptions. 0.20-security is
> obviously entirely different, but as yet remains undefined (it's way more
> than just "security").
>
> -Todd
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
>


-- 
Todd Lipcon
Software Engineer, Cloudera

--00032555903e920d9604a2faa044--