hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt Foley (Issue Comment Edited) (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Edited] (HADOOP-8037) Binary tarball does not preserve platform info for native builds, and fails to provide needed symlinks for libhadoop.so
Date Wed, 08 Feb 2012 10:27:01 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203442#comment-13203442
] 

Matt Foley edited comment on HADOOP-8037 at 2/8/12 10:26 AM:
-------------------------------------------------------------

The symlinks are now good!  However, the rpms are retaining some paths they shouldn't: {code}
$ rpm -qlp hadoop-1.0.1-1.i386.rpm|grep libhadoop
/usr/c++/Linux-i386-32/lib/libhadooppipes.a
/usr/c++/Linux-i386-32/lib/libhadooputils.a
/usr/lib/libhadoop.a
/usr/lib/libhadoop.la
/usr/lib/libhadoop.so
/usr/lib/libhadoop.so.1
/usr/lib/libhadoop.so.1.0.0
/usr/lib/libhadooppipes.a
/usr/lib/libhadooputils.a

$ rpm -qlp hadoop-1.0.1-1.amd64.rpm|grep libhadoop
/usr/c++/Linux-amd64-64/lib/libhadooppipes.a
/usr/c++/Linux-amd64-64/lib/libhadooputils.a
/usr/c++/Linux-i386-32/lib/libhadooppipes.a
/usr/c++/Linux-i386-32/lib/libhadooputils.a
/usr/lib64/libhadoop.a
/usr/lib64/libhadoop.la
/usr/lib64/libhadoop.so
/usr/lib64/libhadoop.so.1
/usr/lib64/libhadoop.so.1.0.0
/usr/lib64/libhadooppipes.a
/usr/lib64/libhadooputils.a
{code}
I think the /usr/c++/<platform>/lib paths shouldn't be in the rpms, only in the bin-tarball.

Also, in the bin-tarball, the libhadooppipes and utils libs are handled inconsistently: {code}
$ tar ztf hadoop-1.0.1-bin.tar.gz|grep libhadoop
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhadooppipes.a
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhadooputils.a
hadoop-1.0.1/c++/Linux-i386-32/lib/libhadooppipes.a
hadoop-1.0.1/c++/Linux-i386-32/lib/libhadooputils.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.la
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1.0.0
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadooppipes.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadooputils.a
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.a
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.la
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1.0.0
{code}
I think the files {code}
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadooppipes.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadooputils.a
{code}
shouldn't be there.

The source tarball looked fine.

Finally, please remove the commented-out line in build.xml: {code} 
<!--<delete dir="${dist.dir}"/> --> {code}

Thanks.

                
      was (Author: mattf):
    The symlinks are now good!  However, the rpms are retaining some paths they shouldn't:
{code}
$ rpm -qlp hadoop-1.0.1-1.i386.rpm|grep libhadoop
/usr/c++/Linux-i386-32/lib/libhadooppipes.a
/usr/c++/Linux-i386-32/lib/libhadooputils.a
/usr/lib/libhadoop.a
/usr/lib/libhadoop.la
/usr/lib/libhadoop.so
/usr/lib/libhadoop.so.1
/usr/lib/libhadoop.so.1.0.0
/usr/lib/libhadooppipes.a
/usr/lib/libhadooputils.a

$ rpm -qlp hadoop-1.0.1-1.amd64.rpm|grep libhadoop
/usr/c++/Linux-amd64-64/lib/libhadooppipes.a
/usr/c++/Linux-amd64-64/lib/libhadooputils.a
/usr/c++/Linux-i386-32/lib/libhadooppipes.a
/usr/c++/Linux-i386-32/lib/libhadooputils.a
/usr/lib64/libhadoop.a
/usr/lib64/libhadoop.la
/usr/lib64/libhadoop.so
/usr/lib64/libhadoop.so.1
/usr/lib64/libhadoop.so.1.0.0
/usr/lib64/libhadooppipes.a
/usr/lib64/libhadooputils.a
{code}
I think the /usr/c++/<platform>/lib paths shouldn't be in the rpms, only in the bin-tarball.

Also, in the bin-tarball, the libhadooppipes and utils libs are handled inconsistently: {code}
$ tar ztf hadoop-1.0.1-bin.tar.gz|grep libhadoop
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhadooppipes.a
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhadooputils.a
hadoop-1.0.1/c++/Linux-i386-32/lib/libhadooppipes.a
hadoop-1.0.1/c++/Linux-i386-32/lib/libhadooputils.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.la
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1.0.0
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadooppipes.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadooputils.a
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.a
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.la
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1.0.0

I think the files {code}
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadooppipes.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadooputils.a
{code}
shouldn't be there.

The source tarball looked fine.

Finally, please remove the commented-out line in build.xml: {code} 
<!--<delete dir="${dist.dir}"/> --> {code}

Thanks.

                  
> Binary tarball does not preserve platform info for native builds, and fails to provide
needed symlinks for libhadoop.so
> -----------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-8037
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8037
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: build
>    Affects Versions: 1.0.1
>            Reporter: Matt Foley
>            Assignee: Giridharan Kesavan
>         Attachments: hadoop-8037.patch
>
>
> The source tarball uses "package" ant target, which includes both sets of native builds
(32 and 64 bit libraries), under subdirectories that are named for the supported platform,
so you can tell what they are.
> The binary tarball uses the "bin-package" ant target, which projects both sets of native
builds into a single directory, stripping out the platform names from the directory paths.
 Since the native built libraries have identical names, only one of each survives the process.
 Afterward, there is no way to know whether they are intended for 32 or 64 bit environments.
> It seems to be done this way as a step toward building the rpm and deb artifacts.  But
the rpms and debs are self-identifying as to the platform they were built for, and contain
only one set of libs each, while the binary tarball isn't.  The binary tarball should have
the same platform-specific subdirectories that the full tarball does; but this means that
the rpm and deb builds have to be more careful about include/exclude specs for what goes into
those artifacts.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message