hadoop-pig-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alan Gates <ga...@yahoo-inc.com>
Subject Restarting discussion on Pig as a TLP
Date Mon, 16 Aug 2010 20:46:06 GMT
Five months ago I started a discussion on whether Pig should become a  
top level project (TLP) at Apache instead of remaining a subproject of  
Hadoop (http://mail-archives.apache.org/mod_mbox/hadoop-pig-dev/201003.mbox/%3C006AEA7C-8829-4788-AD7B-822396FA2E6B@yahoo-inc.com%3E

).  At the time I voted against it (http://mail-archives.apache.org/mod_mbox/hadoop-pig-dev/201003.mbox/%3CF1484964-E774-48B7-9D45-6E57C7B09251@yahoo-inc.com%3E

), as did many others.  However, I would like to restart that  
discussion now.

I gave several reasons for voting against it :

First, I was worried that by loosing our connection to Hadoop, Pig  
would loose its source of new users.  I have since been assured by  
Hadoop members that Pig would be free to keep our tab on their page  
(as Hbase has).  Also, obviously we would still be welcomed at Hadoop  
get togethers such as the various HUGs, Hadoop Summits, etc.  So our  
connection does not seem in danger.

Two, I was concerned that by not being members of the Hadoop community  
we would loose influence with Hadoop.  It is true that Pig developers  
will have to stay active in the Hadoop community, which will put a  
slightly extra burden on them.  But they are already bearing this  
burden, and whether or not the communities are governed by the same or  
separate PMCs will not affect this.

Finally, I said that philosophically it makes sense to me that all  
Hadoop related projects should stay under one umbrella.  This still  
makes sense to me, and I do see this as a downside of Pig moving out  
of Hadoop.

In addition to the above, a few other things have happened over the  
intervening months to cause me to reconsider.  Most importantly, it  
has become clear to me that Pig is operating as if it were a TLP  
inside Hadoop.  We have four members on the Hadoop PMC, which means we  
have sufficient votes to elect our committers and release our products.

Also, several Hadoop PMC members who have long experience in Apache  
projects have made clear to me that they believe Pig is ready to be a  

I was also concerned about diversity in our PMC, since our project is  
Yahoo heavy.  Given that 10 out of 12 committers are Yahoo employees  
we need to work on this.  But we do have experienced committers in  
three different organizations, and I think this gives us sufficient  
base to to work on it as a TLP.

So, in summary, I have switched my view on this from "not yet" to "now  
is a good time".  I think Pig is ready to be a TLP.  We have a  
community of contributors and users that is growing both in numbers  
and in diversity.  We have a strong group of committers who I believe  
are ready to take on leadership of the project and who will benefit  
from being mentored by the larger Apache community.



View raw message