hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Eli Collins <...@cloudera.com>
Subject Re: where do side-projects go in trunk now that contrib/ is gone?
Date Tue, 12 Feb 2013 22:09:18 GMT
I agree that the current place isn't a good one, for both the reasons
you mention on the jira (and because the people maintaining this code
don't primarily work on Hadoop). IMO the SwiftFS driver should live in
the swift source tree (as part of open stack).

I'm not -1 on it living in-tree, it's just not my 1st choice. If you
want to create a top-level directory for 3rd party (read non-local,
non-hdfs file systems) file systems - go for it. It would be an
improvement on the current situation (o.a.h.fs.ftp also brings in
dependencies that most people don't need).  I don't think we need to
come up with a new top-level "kitchen sink" directory to handle all
Hadoop extensions, there are a few well-defined extension points that
can likely be handled independently so logically grouping them
separately makes sense to me (and perhaps we'll decide some extensions
are better in-tree and some not).

On Tue, Feb 12, 2013 at 1:51 PM, Steve Loughran <stevel@apache.org> wrote:
> On 12 February 2013 21:35, Eli Collins <eli@cloudera.com> wrote:
>> To answer your original question, there's already precedent for
>> extension code in the project, eg file system extensions like s3 live
>> in o.a.h.fs. The directory depends on the type of extension (eg 3rd
>> party codec integration lives in o.a.h.io.compress). Probably best to
>> discuss the particular case.
> I know that s3n & S3 are in the main JAR, but I don't think that should be
> copied without good reason, because it's also added another dependency
> (jets3t) to everything.
> The case of where to put SwiftFS driver has cropped up in HADOOP-8545, but
> I don't think a single JIRA is the place to define policy like this
> -hopefully general is.
> -Steve

View raw message