hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (3980)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: [DISCUSS] Additional maintenance releases for Hadoop 2.y versions
Date Wed, 15 Jul 2015 22:09:06 GMT
+1 Chris is right.

++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Chief Architect
Instrument Software and Science Data Systems Section (398)
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 168-519, Mailstop: 168-527
Email: chris.a.mattmann@nasa.gov
WWW:  http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Associate Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++





-----Original Message-----
From: Chris Douglas <cdouglas@apache.org>
Reply-To: "common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>
Date: Wednesday, July 15, 2015 at 3:07 PM
To: "common-dev@hadoop.apache.org" <common-dev@hadoop.apache.org>
Cc: dev <dev@hbase.apache.org>
Subject: Re: [DISCUSS] Additional maintenance releases for Hadoop 2.y
versions

>On Wed, Jul 15, 2015 at 2:07 PM, Sean Busbey <busbey@cloudera.com> wrote:
>> Alternatively, why not appoint a Release Manager for the minor release
>>line
>> and then allow them to arbitrate when there's disagreement about
>>inclusion?
>> This has worked well in the HBase community.
>
>
>Release managers aren't appointed in Hadoop. Any committer can RM a
>release branch and encourage others to help with it. An RM can set the
>bar arbitrarily, but an RC only becomes a release when a majority of
>PMC votes approve it in a VOTE. -C
>
>On Wed, Jul 15, 2015 at 2:07 PM, Sean Busbey <busbey@cloudera.com> wrote:
>> Why not just include all backwards compatible bug fixes?
>>
>> Alternatively, why not appoint a Release Manager for the minor release
>>line
>> and then allow them to arbitrate when there's disagreement about
>>inclusion?
>> This has worked well in the HBase community.
>>
>> On Wed, Jul 15, 2015 at 3:49 PM, Karthik Kambatla <kasha@cloudera.com>
>> wrote:
>>
>>> As I proposed in the other thread, how about we adopting the following
>>> model:
>>>
>>> x.y.1 releases have all Blocker, Critical, Major bug fixes applied to
>>>the
>>> next minor release.
>>> x.y.2 releases have all Blocker, Critical bug fixes applied to the next
>>> minor release.
>>> x.y.3 releases have all Blocker bug fixes applied to next minor
>>>release.
>>>
>>> Here I am assuming there are no security-fix-only or other urgent
>>>releases.
>>>
>>> We could apply this approach for 2.7.x onwards, and do an adhoc 2.6
>>> release.
>>>
>>> On Wed, Jul 15, 2015 at 12:59 PM, Vinod Kumar Vavilapalli <
>>> vinodkv@hortonworks.com> wrote:
>>>
>>> > Yeah, I started a thread while back on this one (
>>> > http://markmail.org/message/sbykjn5xgnksh6wg) and had many offline
>>> > discussions re 2.6.1.
>>> >
>>> > The biggest problem I found offline was about what bug-fixes are
>>> > acceptable and what aren’t for everyone wishing to consume 2.6.1.
>>>Given
>>> the
>>> > number of bug-fixes that went into 2.7.x and into branch-2.8,
>>>figuring
>>> out
>>> > a set of patches that is acceptable for everyone is a huge challenge
>>> which
>>> > kind of stalled my attempts.
>>> >
>>> > Thanks
>>> > +Vinod
>>> >
>>> >
>>> > > On Jul 15, 2015, at 8:57 AM, Sangjin Lee <sjlee0@gmail.com> wrote:
>>> > >
>>> > > Strong +1 for having a 2.6.1 release. I understand Vinod has been
>>> trying
>>> > to
>>> > > get that effort going but it's been stalled a little bit. It would
>>>be
>>> > good
>>> > > to rekindle that effort.
>>> > >
>>> > > Companies with big hadoop 2.x deployments (including mine) have
>>>always
>>> > > tried to stabilize a 2.x release by testing/collecting/researching
>>> > critical
>>> > > issues on the release. Each would come up with its own set of
>>>fixes to
>>> > > backport. We would also communicate it via offline channels.
>>>During the
>>> > > hadoop summit, we thought it would be great if we all came
>>>together and
>>> > > create a public stability/bugfix release on top of 2.x (2.6.1 for
>>>2.6
>>> for
>>> > > example) with all the critical issues fixed.
>>> > >
>>> > > Thanks,
>>> > > Sangjin
>>> > >
>>> > >
>>> > > On Tue, Jul 14, 2015 at 10:42 PM, Tsuyoshi Ozawa <ozawa@apache.org>
>>> > wrote:
>>> > >
>>> > >> Thank you for the notification. Trying to back port bug fixes.
>>> > >>
>>> > >> - Tsuyoshi
>>> > >>
>>> > >> On Wed, Jul 15, 2015 at 3:45 AM, Sean Busbey <busbey@cloudera.com>
>>> > wrote:
>>> > >>> Hi Hadoopers!
>>> > >>>
>>> > >>> Over in HBase we've been discussing the impact of our
>>>dependencies on
>>> > our
>>> > >>> downstream users. As our most fundamental dependency, Hadoop
>>>plays a
>>> > big
>>> > >>> role in the operational cost of running an HBase instance.
>>> > >>>
>>> > >>> Currently the HBase 1.y release line supports Hadoop 2.4, 2.5,
>>>and
>>> > >> 2.6[1].
>>> > >>> We don't drop Hadoop minor release lines in minor releases
so we
>>>are
>>> > >>> unlikely remove anything from this set until HBase 2.0, probably
>>>at
>>> the
>>> > >> end
>>> > >>> of 2015 / start of 2016 (and currently we plan to continue
>>>supporting
>>> > at
>>> > >>> least 2.4 for HBase 2.0 [2]). Lately we've been discussing
>>>updating
>>> our
>>> > >>> shipped binaries to Hadoop 2.6, following some stability testing
>>>by
>>> > part
>>> > >> of
>>> > >>> our community[3]. Unfortunately, 2.6.0 in particular has a
>>>couple of
>>> > bugs
>>> > >>> that could destroy HBase clusters should users decide to turn
on
>>>HDFS
>>> > >>> encryption[4]. Our installation instructions tell folks to
>>>replace
>>> > these
>>> > >>> jars with the version of Hadoop they are actually running,
but
>>>not
>>> all
>>> > >>> users follow those instructions so we want to minimize the
pain
>>>for
>>> > them.
>>> > >>>
>>> > >>> Regular maintenance releases are key to keeping operational
>>>burdens
>>> low
>>> > >> for
>>> > >>> our downstream users; we don't want them to be forced to choose
>>> between
>>> > >>> living with broken systems and stomaching the risk of upgrades
>>>across
>>> > >>> minor/major version numbers. Looking back over the three
>>> aforementioned
>>> > >>> Hadoop versions, 2.6 hasn't had a patch release since 2.6.0
came
>>>out
>>> in
>>> > >> Nov
>>> > >>> 2014, when 2.5 had its last patch release as well. Hadoop 2.4
>>>looks
>>> to
>>> > >> be a
>>> > >>> year without a release[5]. On our discussion of shipping Hadoop
>>>2.6
>>> > >>> binaries, one of your PMC members mentioned that with continued
>>>work
>>> on
>>> > >> the
>>> > >>> 2.7 line y'all weren't planning any additional releases of
the
>>> earlier
>>> > >>> minor versions[6].
>>> > >>>
>>> > >>> The HBase community requests that Hadoop pick up making
>>>bug-fix-only
>>> > >> patch
>>> > >>> releases again on a regular schedule[7]. Preferably on the
2.6
>>>line
>>> and
>>> > >>> preferably monthly. We realize that given the time gap since
>>>2.6.0 it
>>> > >> will
>>> > >>> likely take a big to get 2.6.1 together, but after that it
should
>>> take
>>> > >> much
>>> > >>> less effort to continue.
>>> > >>>
>>> > >>> [1]: http://hbase.apache.org/book.html#hadoop
>>> > >>> [2]: http://s.apache.org/ReP
>>> > >>> [3]: HBASE-13339
>>> > >>> [4]: HADOOP-11674 and HADOOP-11710
>>> > >>> [5]: http://hadoop.apache.org/releases.html
>>> > >>> [6]: http://s.apache.org/MTY
>>> > >>> [7]: http://s.apache.org/ViP
>>> > >>>
>>> > >>> --
>>> > >>> Sean
>>> > >>
>>> >
>>> >
>>>
>>>
>>> --
>>> Karthik Kambatla
>>> Software Engineer, Cloudera Inc.
>>> --------------------------------------------
>>> http://five.sentenc.es
>>>
>>
>>
>>
>> --
>> Sean

Mime
View raw message