hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (388J)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: [VOTE] - Establish YARN as a sub-project of Apache Hadoop
Date Thu, 16 Aug 2012 20:41:22 GMT
Hi Eli,

On Aug 16, 2012, at 1:21 PM, Eli Collins wrote:
>>> [...snip...]
>> Keeping code in the same repository with a PMC with different sets of
>> permissions in that repository *is* the sign of a distinct community.
>> Doesn't matter if you want the code there together, and allowing patches,
>> and testing and releasing and whatever. Those are technical issues.
>> Having code with *different* rules for *the same* community members
>> is not what I know as "community over code" and the Apache way.
>> And ultimately it's the reason why this email thread won't die. There's
>> an elephant in the room here (pun intended).
> For the PMC the permissions are the same across all the subprojects
> and it is the same community. The question here is around committers.

Trying to distinguish between the 2 (PMC and committers) leads you to this.
I guess the Hadoop PMC is interested in the overhead of managing different
lists of people with different permissions who release different things. That
seems like an awfully lot of worthless overhead to me, and discussions 
around topics that pretty much wouldn't matter at all if you adopted a flatter
organizational structure. 

See 2 of my favorite quotes from former ASF Director Hen Yandell:


So whether you are talking about "PMC" or "committers" here, is really
moot on whether or not this is a community issue. It is. No matter how
you spin it.

> We already have a working model where the various subprojects have
> their own committers and people are trusted to commit across project
> boundaries when it makes sense.

Depends on what you consider to be working. Sure if the Hadoop PMC
wants to take on the overhead in a centralized fashion that is normally
distributed amongst projects and their own distinct PMCs that's fine.
Just doesn't seem to be a scalable solution that's all. Because if you
look back at most of the Apache projects that did this (e.g., umbrella 
projects) they don't work out. That's b/c having a single PMC that has
merit across all of the sub-projects usually doesn't work because not
every person on that PMC rightly should have an equal VOTE on 
those sub-projects that they have no merit in. This can easily be
the case when you have things like e.g., YARN, or new things come
along that you guys yourself try and bless instead of just realizing
that it's a distinct community. 

> I think everyone is in agreement that
> yarn, like the other subprojects, will have it's own set of
> committers, the only open question here as I understand it is whether
> or not we grandfather in the existing MR committers (since that's
> where YARN used to live).  

If there's even a discussion about this, there's an issue. You guys
think this discussion is new and what you arent' realizing is that
we've had it before. In MANY Apache projects. Over MANY years.

So what's new to you is not new to me. And what's frustrating is watching
you guys waste time debating and discussing something that's already
been solved before. MANY times.

> Ie I don't see this as a major elephant in
> the room, just need to decide between Arun's proposal where we
> actively exclude these people or Tom's proposal where we don't.
> Perhaps we should just vote.

VOTEing to solve issues at Apache is also unfortunately the sign
of a community issue. Most healthy communities I see at the Foundation
discuss, reach consensus, and VOTEs are merely a formality putting pen
and paper to an existing consensus. 

Anyhoo, I'll return back to the sidelines. I just thought after a few days
of this, I'd pipe up b/c I think you guys can VOTE however many times
you want, and create as many sub-whosamuwwhatsits as many times
as you want with different committers and whatever else, I just think
there could be an easier way. And it doesn't have to be re-invented here.

//back to sidelines


Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA

View raw message