hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Boudnik <...@boudnik.org>
Subject Re: [DISCUSS] Move common, hdfs, mapreduce contrib components to apache-extras.org or elsewhere
Date Mon, 31 Jan 2011 05:24:34 GMT
While this seems to be a very good and long awaited advancement (btw
thanks to Eric Sammer for making a first project on the move out of
MRUnit!) I have a concern about lack of Git support on
http://apache-extras.org. I am sure that hosting decision was deeply
considered by Apache board (or the whole Apache community) however
SVN/Mercurial seems like a real drag for most of Hadoop developers.

Basically, if contribs are moved to apache-extras.org then we are
likely to face the same situation as with the core Hadoop: lotta
development is done in personal Git repos forked from a Git/SVN mirror
and then is committed back to the man SVN repository creating an extra

Shall we not dictate a location of contrib projects once they are
moved of Hadoop? If ppl feel like they are better be served by GitHub
perhaps they should have an option to get hosted there?
  Take care,
Konstantin (Cos) Boudnik

On Sun, Jan 30, 2011 at 19:42, Nigel Daley <ndaley@mac.com> wrote:
> Folks,
> Now that http://apache-extras.org is launched (https://blogs.apache.org/foundation/entry/the_apache_software_foundation_launches)
I'd like to start a discussion on moving contrib components out of common, mapreduce, and
> These contrib components complicate the builds, cause test failures that nobody seems
to care about, have releases that are tied to Hadoop's long release cycles, etc.  Most folks
I've talked with agree that these contrib components would be better served by being pulled
out of Hadoop and hosted elsewhere. The new apache-extras code hosting site seems like a natural
*default* location for migrating these contrib projects.  Perhaps some should graduate from
contrib to src (ie from contrib to core of the project they're included in).  If folks agree,
we'll need to come up with a mapping of contrib component to it's final destination and file
a jira.
> Here are the contrib components by project (hopefully I didn't miss any).
> Common Contrib:
>  failmon
>  hod
>  test
> MapReduce Contrib:
>  capacity-scheduler -- move to MR core?
>  data_join
>  dynamic-scheduler
>  eclipse-plugin
>  fairscheduler -- move to MR core?
>  gridmix
>  index
>  mrunit
>  mumak
>  raid
>  sqoop
>  streaming -- move to MR core?
>  vaidya
>  vertica
> HDFS Contrib:
>  fuse-dfs
>  hdfsproxy
>  thriftfs
> Cheers,
> Nige

View raw message