hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Devaraj Das <d...@yahoo-inc.com>
Subject Re: Release compatibility was Re: [VOTE] Release candidate
Date Wed, 11 May 2011 05:24:34 GMT
Just so that everyone is on the same page w.r.t the compatibility between 20.2 & 20.203
(don't think this is documented anywhere yet)..

The aim of the team working on Hadoop Security at Yahoo! was to make Hadoop *secure*, and
with *minimal* disruption to existing apps. I can't think of any major user-facing incompatibilities
other than the users having to run a 'kinit' when they are working with a secure Hadoop cluster
(of course the admins need to do more work in order to set up a secure cluster). Also, security
can switched off, and all the other enhancements (job limits, etc.) are still available..
As per users/Operations/Solutions at Yahoo!, 20.security was one of the smoothest upgrades

On 5/10/11 2:28 PM, "Todd Lipcon" <todd@cloudera.com> wrote:

On Tue, May 10, 2011 at 12:41 PM, Scott Carey <scott@richrelevance.com>wrote:

> As an observer, this is a very important observation.  Sure, the default
> is that dot releases are bugfix-onl.  But exceptions to these rules are
> sometimes required and often beneficial to the health of the project.
> Performance enhancements, minor features, and other items are sometimes
> very low risk and the barrier to getting them to users earlier should be
> lower.

I agree whole-heartedly.

> These issues are the sort of things that get into non-Apache releases
> quickly and drive the community away from the Apache release.  Its been
> well proven through those vehicles that back-porting minor features and
> improvements from trunk to an old release can be done safely.

However, one shouldn't understate the difficulty of agreeing on the
risk-reward tradeoff here. While risk is mostly technical, reward may vary
widely based on the userbase or organization.

For example, everyone would agree that security was a very risky feature to
add to 20, with known backward compatibilities and a lot of fallout. For
some people (both CDH and YDH), the security features were an absolute
necessity on a tight timeline, so the risk-reward decision was clear -- I've
heard from many users, though, that they saw none of the reward from
security and wished they hadn't had to endure the resulting changes and bugs
within the 0.20 series.

Another example is the 0.20-append patch series, which is indispensable for
the HBase community but seen as overly risky by those who do not use HBase.

So, while I'm in favor of "sustaining" release series like 0.20-security in
theory, I also think we need a clear inclusion criteria for such branches.
As I said in a previous email, the criteria used to be "low risk compatible
bug fixes only" with a vote process for any exceptions. 0.20-security is
obviously entirely different, but as yet remains undefined (it's way more
than just "security").

Todd Lipcon
Software Engineer, Cloudera

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message