pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alan Gates <ga...@hortonworks.com>
Subject Re: Next Pig release proposal
Date Fri, 21 Oct 2011 18:00:44 GMT

On Oct 20, 2011, at 4:58 PM, Santhosh Srinivasan wrote:

> Ref: http://tinyurl.com/4ng8upa (last discussion on 1.0)
> How far have we progressed from our last discussion in March. There was no consensus
on the 1.0 release. Opinions ranged from having more releases to bake in the maturity of the
new parser and logical plan changes to compatibility with Hadoop API (was compared to Social
Security - a very hot topic these days).
> My concerns were around Hadoop API stability. I have heard that the APIs will not be
stable for at least 1 year. This is taking me away from the Hadoop API stability factor (They
passed healthcare in that duration. Really!) Do we want compatibility with 0.23 as a gating
factor - not sure if this is anywhere close to getting done in the near future. Will we support
append (0.20.205)?

As Thejas points out, we'll be supporting both for some time.  Given the size of the Hadoop
user base, even assuming 0.23 is an instant success, the 0.20.x line will be around for two
years or more.   Pig releases will have to work with both 20 and 23 in this time.  So I agree
that support for 0.23 is not related to what we call our release.  Also, some contributors
seem interested in supporting Pig for Hadoop 0.22 (see https://issues.apache.org/jira/browse/PIG-2277),
so we need to figure out how we make releases tailored to specific version of Hadoop.

I agree we have not met Santhosh's criteria of Hadoop having a stable API, and I don't think
we will for a couple more years.  I believe we have met Dmitriy's criteria of letting the
0.9 changes bake for 6 months and not doing any more potentially destabilizing rewrites.

So, here's my +1 for the branch and for going 1.0.


> Btw, Hbase has been doing 0.90.1, 0.90.2, etc. So we can take a look at this option too.
> Santhosh
> -----Original Message-----
> From: Olga Natkovich [mailto:olgan@yahoo-inc.com] 
> Sent: Thursday, October 20, 2011 4:40 PM
> To: dev@pig.apache.org
> Subject: Next Pig release proposal
> Hi,
> Here is what I propose we do for the next Pig release:
> (1)    Branch early next week - we have major features  and many bug fixes in and will
be fixing remaining bugs on the branch
> (2)    Publish the release by 11/15 - that will give us a couple of weeks to stabilize
the branch and get last minute bug fixes in
> (3)    Make this release a 1.0 release. Reasons to go for 1.0 and not 0.10
> a.       This release has minimal number of features and was focused on code stabilization
and bug fixes. We believe it will be a stable release
> b.      It has no major interface changes
> c.       Going from 0.10 to 1.0 would be very confusing
> Comments?
> Olga

View raw message