hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Santhosh Srinivasan <...@yahoo-inc.com>
Subject RE: Restarting discussion on Pig as a TLP
Date Tue, 17 Aug 2010 18:28:17 GMT

Very nice! I see that opinion has changed within a couple of quarters :) I am pasting references
to my responses to Alan's initial proposal to make Pig a TLP. I am all for making Pig a TLP.


1. http://mail-archives.apache.org/mod_mbox/hadoop-pig-dev/201004.mbox/%3C088A0B616C8C1D4787DD686C6922A72A03161728@SNV-EXVS10.ds.corp.yahoo.com%3E
2. http://mail-archives.apache.org/mod_mbox/hadoop-pig-dev/201004.mbox/%3C088A0B616C8C1D4787DD686C6922A72A03161767@SNV-EXVS10.ds.corp.yahoo.com%3E

-----Original Message-----
From: Alan Gates [mailto:gates@yahoo-inc.com] 
Sent: Monday, August 16, 2010 1:46 PM
To: pig-dev@hadoop.apache.org
Subject: Restarting discussion on Pig as a TLP

Five months ago I started a discussion on whether Pig should become a top level project (TLP)
at Apache instead of remaining a subproject of Hadoop (http://mail-archives.apache.org/mod_mbox/hadoop-pig-dev/201003.mbox/%3C006AEA7C-8829-4788-AD7B-822396FA2E6B@yahoo-inc.com%3E
).  At the time I voted against it (http://mail-archives.apache.org/mod_mbox/hadoop-pig-dev/201003.mbox/%3CF1484964-E774-48B7-9D45-6E57C7B09251@yahoo-inc.com%3E
), as did many others.  However, I would like to restart that discussion now.

I gave several reasons for voting against it :

First, I was worried that by loosing our connection to Hadoop, Pig would loose its source
of new users.  I have since been assured by Hadoop members that Pig would be free to keep
our tab on their page (as Hbase has).  Also, obviously we would still be welcomed at Hadoop
get togethers such as the various HUGs, Hadoop Summits, etc.  So our connection does not seem
in danger.

Two, I was concerned that by not being members of the Hadoop community we would loose influence
with Hadoop.  It is true that Pig developers will have to stay active in the Hadoop community,
which will put a slightly extra burden on them.  But they are already bearing this burden,
and whether or not the communities are governed by the same or separate PMCs will not affect

Finally, I said that philosophically it makes sense to me that all Hadoop related projects
should stay under one umbrella.  This still makes sense to me, and I do see this as a downside
of Pig moving out of Hadoop.

In addition to the above, a few other things have happened over the intervening months to
cause me to reconsider.  Most importantly, it has become clear to me that Pig is operating
as if it were a TLP inside Hadoop.  We have four members on the Hadoop PMC, which means we
have sufficient votes to elect our committers and release our products.

Also, several Hadoop PMC members who have long experience in Apache projects have made clear
to me that they believe Pig is ready to be a TLP.

I was also concerned about diversity in our PMC, since our project is Yahoo heavy.  Given
that 10 out of 12 committers are Yahoo employees we need to work on this.  But we do have
experienced committers in three different organizations, and I think this gives us sufficient
base to to work on it as a TLP.

So, in summary, I have switched my view on this from "not yet" to "now is a good time".  I
think Pig is ready to be a TLP.  We have a community of contributors and users that is growing
both in numbers and in diversity.  We have a strong group of committers who I believe are
ready to take on leadership of the project and who will benefit from being mentored by the
larger Apache community.



View raw message