hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Mattmann, Chris A (388J)" <chris.a.mattm...@jpl.nasa.gov>
Subject Re: [DISCUSS] Move common, hdfs, mapreduce contrib components to apache-extras.org or elsewhere
Date Thu, 10 Feb 2011 06:53:33 GMT
Hi Nige,

Thanks. I think in fact it does directly relate -- to use your verbage below -- not ever svn
remove <dir> requires a [DISCUSS] thread before svn removing it, right?

Hence, my knee-jerk reaction on reading this thread is that contrib modules are = contrib
modules in other umbrella projects -- some are small, and likely yes should be part of the
core of Hadoop; others are not, and are in fact, "mini projects" that have been baking up
in Hadoop for a while. In the case of the former, I agree, +1, those should be moved as part
of the Hadoop core "project". In the case of the latter, I do not agree that simply the Hadoop
project should create a wiki page and declare by VOTE that there won't be anymore development
on them. In fact, they are perfect candidates for moving to the Attic where someone besides
the Hadoop PMC might want to pick up on them at a later point in time.

There is no hard and fast rule for the size of a TLP too btw: utilities that run on top of
Hadoop could go through e.g., Incubation, and eventually graduate to TLPs.

Cheers,
Chris

On Feb 9, 2011, at 10:35 PM, Nigel Daley wrote:

> Hi Chris,
> 
> You use the word 'project' over and over.  These contrib modules are not Apache projects.
 They are source code directories within the various Hadoop subprojects. Reading thru attic.apache.org,
I don't see how it directly relates.  Surely not every 'svn remove <dir>' should be
replaced with a move to attic.apache.org.  
> 
> Cheers,
> Nige
> 
> 
> On Feb 9, 2011, at 7:37 PM, Mattmann, Chris A (388J) wrote:
> 
>> Hi Nigel,
>> 
>> My 2 cents -- why is Hadoop re-creating its own mini-Attic? The point of the umbrella
project shakedown over the past year at Apache is to stop projects from recreating things
like the Incubator (and the Attic) inside of the project.
>> 
>> One suggestion: for your [VOTE] thread that you are calling -- if the result is to
attic the project -- move it to the "real" Apache Attic, via a board resolution (you could
do a single board resolution from the Hadoop PMC to the Apache Board containing the set of
projects to Attic). 
>> 
>> Cheers,
>> Chris
>> 
>> On Feb 9, 2011, at 9:11 AM, Nigel Daley wrote:
>> 
>>> After considering the feedback, I will move forward with calling a separate vote
for each contrib (or small groups of contribs). The vote will ask the PMC to abandon the given
contrib or move it to core.  For those we agree to abandon, I will setup an Attic wiki that
will point to the last SVN revision of the contrib.  I will also start a Related Projects
wiki (if we don't already have one) with pointers to the contrib modules that folks have volunteered
to keep developing elsewhere.
>>> 
>>> Cheers,
>>> Nige
>>> 
>>> On Jan 30, 2011, at 7:42 PM, Nigel Daley wrote:
>>> 
>>>> Folks,
>>>> 
>>>> Now that http://apache-extras.org is launched (https://blogs.apache.org/foundation/entry/the_apache_software_foundation_launches)
I'd like to start a discussion on moving contrib components out of common, mapreduce, and
hdfs.  
>>>> 
>>>> These contrib components complicate the builds, cause test failures that
nobody seems to care about, have releases that are tied to Hadoop's long release cycles, etc.
 Most folks I've talked with agree that these contrib components would be better served by
being pulled out of Hadoop and hosted elsewhere. The new apache-extras code hosting site seems
like a natural *default* location for migrating these contrib projects.  Perhaps some should
graduate from contrib to src (ie from contrib to core of the project they're included in).
 If folks agree, we'll need to come up with a mapping of contrib component to it's final destination
and file a jira.
>>>> 
>>>> Here are the contrib components by project (hopefully I didn't miss any).
>>>> 
>>>> Common Contrib:
>>>> failmon
>>>> hod
>>>> test
>>>> 
>>>> 
>>>> MapReduce Contrib:
>>>> capacity-scheduler -- move to MR core?
>>>> data_join
>>>> dynamic-scheduler
>>>> eclipse-plugin
>>>> fairscheduler -- move to MR core?
>>>> gridmix
>>>> index
>>>> mrunit
>>>> mumak
>>>> raid
>>>> sqoop
>>>> streaming -- move to MR core?
>>>> vaidya
>>>> vertica
>>>> 
>>>> 
>>>> HDFS Contrib:
>>>> fuse-dfs
>>>> hdfsproxy
>>>> thriftfs
>>>> 
>>>> 
>>>> Cheers,
>>>> Nige
>>> 
>> 
>> 
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Chris Mattmann, Ph.D.
>> Senior Computer Scientist
>> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
>> Office: 171-266B, Mailstop: 171-246
>> Email: chris.a.mattmann@nasa.gov
>> WWW:   http://sunset.usc.edu/~mattmann/
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> Adjunct Assistant Professor, Computer Science Department
>> University of Southern California, Los Angeles, CA 90089 USA
>> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>> 
> 


++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Chris Mattmann, Ph.D.
Senior Computer Scientist
NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
Office: 171-266B, Mailstop: 171-246
Email: chris.a.mattmann@nasa.gov
WWW:   http://sunset.usc.edu/~mattmann/
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Adjunct Assistant Professor, Computer Science Department
University of Southern California, Los Angeles, CA 90089 USA
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++


Mime
View raw message