hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt Foley (Issue Comment Edited) (JIRA)" <j...@apache.org>
Subject [jira] [Issue Comment Edited] (HADOOP-8037) Binary tarball does not preserve platform info for native builds, and RPMs fail to provide needed symlinks for libhadoop.so
Date Wed, 08 Feb 2012 20:52:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-8037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203973#comment-13203973
] 

Matt Foley edited comment on HADOOP-8037 at 2/8/12 8:52 PM:
------------------------------------------------------------

The RPMs now appear perfect.  However, the bin-tarball still has a few 64-bit-specific files
that shouldn't be there. See lines flagged with {{"<==="}} below: {code}
$ tar ztf hadoop-1.0.1.tar.gz|grep libhdfs|grep -v /src/ |grep -v /docs/
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.la
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.so
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.so.0
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.so.0.0.0
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.la
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.so
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.so.0
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.so.0.0.0

$ tar ztf hadoop-1.0.1-bin.tar.gz|grep libhdfs
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.la
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.so
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.so.0
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.so.0.0.0
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.la
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.so
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.so.0
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.so.0.0.0
hadoop-1.0.1/lib/native/Linux-amd64-64/libhdfs.la       <===
hadoop-1.0.1/lib/native/Linux-amd64-64/libhdfs.so       <===
hadoop-1.0.1/lib/native/Linux-amd64-64/libhdfs.so.0     <===
hadoop-1.0.1/lib/native/Linux-amd64-64/libhdfs.so.0.0.0 <===
{code}
and {code}
$ tar ztf hadoop-1.0.1.tar.gz|grep libhadoop|grep -v /src/ |grep -v /docs/
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhadooppipes.a
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhadooputils.a
hadoop-1.0.1/c++/Linux-i386-32/lib/libhadooppipes.a
hadoop-1.0.1/c++/Linux-i386-32/lib/libhadooputils.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.la
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1.0.0
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.a
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.la
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1.0.0

$ tar ztf hadoop-1.0.1-bin.tar.gz|grep libhadoop
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhadooppipes.a
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhadooputils.a
hadoop-1.0.1/c++/Linux-i386-32/lib/libhadooppipes.a
hadoop-1.0.1/c++/Linux-i386-32/lib/libhadooputils.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.la
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1.0.0
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadooppipes.a   <===
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadooputils.a   <===
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.a
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.la
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1.0.0
{code}

                
      was (Author: mattf):
    The RPMs now appear perfect.  However, the bin-tarball still has {{hadoop-1.0.1/lib/native/Linux-amd64-64/libhdfs.*}}
files, and {{hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop{pipes,utils}.a}} files, that
shouldn't be there: {code}
$ tar ztf hadoop-1.0.1.tar.gz|grep libhdfs|grep -v /src/ |grep -v /docs/
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.la
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.so
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.so.0
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.so.0.0.0
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.la
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.so
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.so.0
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.so.0.0.0

$ tar ztf hadoop-1.0.1-bin.tar.gz|grep libhdfs
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.la
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.so
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.so.0
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.so.0.0.0
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.la
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.so
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.so.0
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.so.0.0.0
hadoop-1.0.1/lib/native/Linux-amd64-64/libhdfs.la       <===
hadoop-1.0.1/lib/native/Linux-amd64-64/libhdfs.so       <===
hadoop-1.0.1/lib/native/Linux-amd64-64/libhdfs.so.0     <===
hadoop-1.0.1/lib/native/Linux-amd64-64/libhdfs.so.0.0.0 <===
{code}
and {code}
$ tar ztf hadoop-1.0.1.tar.gz|grep libhadoop|grep -v /src/ |grep -v /docs/
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhadooppipes.a
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhadooputils.a
hadoop-1.0.1/c++/Linux-i386-32/lib/libhadooppipes.a
hadoop-1.0.1/c++/Linux-i386-32/lib/libhadooputils.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.la
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1.0.0
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.a
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.la
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1.0.0

$ tar ztf hadoop-1.0.1-bin.tar.gz|grep libhadoop
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhadooppipes.a
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhadooputils.a
hadoop-1.0.1/c++/Linux-i386-32/lib/libhadooppipes.a
hadoop-1.0.1/c++/Linux-i386-32/lib/libhadooputils.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.la
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1.0.0
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadooppipes.a   <===
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadooputils.a   <===
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.a
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.la
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1.0.0
{code}
See lines flagged with {{"<==="}} above.
                  
> Binary tarball does not preserve platform info for native builds, and RPMs fail to provide
needed symlinks for libhadoop.so
> ---------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-8037
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8037
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: build
>    Affects Versions: 1.0.1
>            Reporter: Matt Foley
>            Assignee: Giridharan Kesavan
>         Attachments: hadoop-8027-1.patch, hadoop-8037-1.patch, hadoop-8037.patch
>
>
> The source tarball uses "package" ant target, which includes both sets of native builds
(32 and 64 bit libraries), under subdirectories that are named for the supported platform,
so you can tell what they are.
> The binary tarball uses the "bin-package" ant target, which projects both sets of native
builds into a single directory, stripping out the platform names from the directory paths.
 Since the native built libraries have identical names, only one of each survives the process.
 Afterward, there is no way to know whether they are intended for 32 or 64 bit environments.
> It seems to be done this way as a step toward building the rpm and deb artifacts.  But
the rpms and debs are self-identifying as to the platform they were built for, and contain
only one set of libs each, while the binary tarball isn't.  The binary tarball should have
the same platform-specific subdirectories that the full tarball does; but this means that
the rpm and deb builds have to be more careful about include/exclude specs for what goes into
those artifacts.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message