hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (388J)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: [DISCUSS] Spin out MR, HDFS and YARN as their own TLPs and disband Hadoop umbrella project
Date Wed, 29 Aug 2012 23:29:27 GMT
Hi Todd,

On Aug 29, 2012, at 2:18 PM, Todd Lipcon wrote:

> Have we not learned our lessons from the last attempts to split?
> The issues in our community, which I think Chris is referring to, do
> not generally revolve around project boundaries. It's not the case
> that the HDFS community wants to go one way and the MR/YARN community
> wants to go another, and we get into a conflict around it. If it were,
> then splitting into separate TLPs would make a ton of sense.

You're right, it's not project boundaries, it's poor community behavior, 
and general umbrella-project-ness.

One aspect I've seen is that exclusivity of allowing people to become
PMC members on the project, and the separation of PMC from C. 
Other things I've seen are the use of technical justifications or complexity
issues as an excuse for the exclusivity, as an excuse for drawing boundaries
between project committers and PMC members, and then between specific
products that the project and community as a whole releases, and finally
other things I've seen include external interests influencing the way that 
business is done around here (need for releases in downstream companies,
or projects driving upstream, Apache decisions, which are supposed to be
independent of any lone company, or set of companies -- it's individuals here
that do the work).

The above is not a discrete thing that's happened once, or twice, or that
happened three times, but was fixed later. It's never been fixed. 

> Instead, the issues are usually _within_ a component. So, if we split
> into 3 TLPs, then we'll just have 3 TLPs, each of which is just as
> contentious as before.

I doubt that. Creating TLPs either directly by going to the board, or
via going to the Incubator should involve a set of members of the 
committee (PMC) that desire to work together; that ideally trust one another; that
are inclusive to others who jump on the list and discuss things; and that
collect these principles into the "Apache way", and build and deliver software at
no cost to the public via this Foundation.

Currently, the Apache Hadoop project isn't doing that. Something needs 
to be done to fix it. Just because an attempt to split the projects in the past
didn't work doesn't mean that the Hadoop community should just accept 
"this is a popular project; it's going to be contentious; nothing to see here

It's more than that.

> Let's just embrace contention as a fact of life on a high-profile
> high-stakes project and get back to work.

-1 to that. Apache projects shouldn't be contentious, whether you are a billion dollar
industry like Hadoop, or whether you are the US govt, or whether you are Joe Blow, 
Mom and Pop, building software to deliver to food truck vendors. It doesn't matter.

> I wasted nearly a month undoing the mess of the last attempt, and I
> don't see why this time it would go any better. -1 from my perspective
> on splitting again at this point. Perhaps if we get to the point that
> we're never making cross-project commits it makes sense, but we're not
> there still.

Again, technical issues cited for community problems. *there are not technical issues*.


Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA

View raw message