hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-9971) Pack hadoop compress native libs and upload it to maven for other projects to depend on
Date Tue, 17 Sep 2013 17:48:51 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-9971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13769732#comment-13769732

Chris Nauroth commented on HADOOP-9971:

I'm curious how this change would interact with Apache release management.  I'm not a release
manager though, so I'm not fully qualified to make a decision.  Can a current release manager
please comment?  [~mattf], maybe you could take a look?

My concern is that the patch seems to imply a need for an Apache release artifact per supported
platform architecture.  Do we have a formally defined list of those supported platforms? 
In practice, we've seen various *nixes: multiple Linux distros, BSD, Solaris, AIX.  There
is also hadoop.dll and winutils.exe for Windows.

This also would seem to drive additional complexity into the mvn build.  How would we handle
uploading multiple release artifacts for multiple platforms under the same version number,
given that the mvn build is executing on one machine with one specific architecture?  Would
Maven let us incrementally upload different platform-specific artifacts under the same version
number?  Would we need to scatter builds to multiple platform slaves and then gather them
for the upload?  Either way implies Apache infrastructure work to guarantee we have the right
mix of machines.

commons-daemon has the same challenge of mixing Java and native deployment artifacts, and
I believe they no longer publish the native artifacts and only release source.  [~liushaohui],
you mentioned hadoop-snappy.  Do you know how that project handles releases?

Thank you for the patch.  It would be convenient if we can work it out, but I think release
managers are the most qualified to review.

> Pack hadoop compress native libs and upload it to maven for other projects to depend
> ---------------------------------------------------------------------------------------
>                 Key: HADOOP-9971
>                 URL: https://issues.apache.org/jira/browse/HADOOP-9971
>             Project: Hadoop Common
>          Issue Type: Improvement
>            Reporter: Liu Shaohui
>            Priority: Minor
>         Attachments: HADOOP-9971-trunk-v1.diff
> Currently, if other projects like hbase want to using hadoop common native lib, they
must copy the native libs to their distribution, which is not agile. From the idea of hadoop-snappy(http://code.google.com/p/hadoop-snappy),
we can pack the hadoop common native lib and upload it to maven repository for other projects
to depend on.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message