Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Sun, 12 Feb 2012 09:36:59 +0000 (UTC)
From: "Matt Foley (Commented) (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: 
 <2097470535.29296.1329039419656.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <825613320.13754.1328689919661.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (HADOOP-8037) Binary tarball does not preserve
 platform info for native builds, and RPMs fail to provide needed symlinks
 for libhadoop.so
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-8037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13206382#comment-13206382 ] 

Matt Foley commented on HADOOP-8037:
------------------------------------

Please review the new patch attached.  It seems to do all the right things.  RPMs and DEBs now use an intermediate bin-tarball with an "${os.arch}" tag (like the packages themselves).  The un-tagged bin-tarball is now multi-platform and retains the structure of the source tarball; it is in fact generated by target "tar", not by target "binary".  Finally, in the 64-bit RPMs and DEBs, the native libs go in "lib64" instead of "lib", as suggested by Giri's previous patch.

Results of the checks are: {code}
file sizes:
-rw-r--r-- 1 hortonfo users 32507490 Feb 12 08:57 hadoop_1.0.1-1_amd64.deb
-rw-r--r-- 1 hortonfo users 32551430 Feb 12 08:55 hadoop-1.0.1-1.amd64.rpm
-rw-r--r-- 1 hortonfo users 36460606 Feb 12 08:50 hadoop_1.0.1-1_i386.deb
-rw-r--r-- 1 hortonfo users 36522879 Feb 12 08:49 hadoop-1.0.1-1.i386.rpm
-rw-r--r-- 1 hortonfo users 33695494 Feb 12 08:54 hadoop-1.0.1-bin.tar.gz
-rw-r--r-- 1 hortonfo users 60790076 Feb 12 08:54 hadoop-1.0.1.tar.gz

file counts:
$ rpm -lqp hadoop-1.0.1-1.amd64.rpm|wc -l ==> 448
$ rpm -lqp hadoop-1.0.1-1.i386.rpm |wc -l ==> 450
$ tar  ztf hadoop-1.0.1-bin.tar.gz |wc -l ==> 383
$ tar  ztf hadoop-1.0.1.tar.gz     |wc -l ==> 6994

$ tar ztf hadoop-1.0.1.tar.gz|grep libhdfs|grep -v /src/ |grep -v /docs/
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.la
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.so
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.so.0
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.so.0.0.0
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.la
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.so
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.so.0
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.so.0.0.0

$ tar ztf hadoop-1.0.1-bin.tar.gz|grep libhdfs
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.la
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.so
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.so.0
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhdfs.so.0.0.0
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.la
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.so
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.so.0
hadoop-1.0.1/c++/Linux-i386-32/lib/libhdfs.so.0.0.0

$ tar ztf hadoop-1.0.1.tar.gz|grep libhadoop|grep -v /src/ |grep -v /docs/
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhadooppipes.a
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhadooputils.a
hadoop-1.0.1/c++/Linux-i386-32/lib/libhadooppipes.a
hadoop-1.0.1/c++/Linux-i386-32/lib/libhadooputils.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.la
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1.0.0
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.a
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.la
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1.0.0

$ tar ztf hadoop-1.0.1-bin.tar.gz|grep libhadoop
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhadooppipes.a
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhadooputils.a
hadoop-1.0.1/c++/Linux-i386-32/lib/libhadooppipes.a
hadoop-1.0.1/c++/Linux-i386-32/lib/libhadooputils.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.la
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1.0.0
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.a
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.la
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1.0.0

$ rpm -qlp hadoop-1.0.1-1.i386.rpm|grep libhadoop
/usr/lib/libhadoop.a
/usr/lib/libhadoop.la
/usr/lib/libhadoop.so
/usr/lib/libhadoop.so.1
/usr/lib/libhadoop.so.1.0.0
/usr/lib/libhadooppipes.a
/usr/lib/libhadooputils.a

$ rpm -qlp hadoop-1.0.1-1.amd64.rpm|grep libhadoop
/usr/lib64/libhadoop.a
/usr/lib64/libhadoop.la
/usr/lib64/libhadoop.so
/usr/lib64/libhadoop.so.1
/usr/lib64/libhadoop.so.1.0.0
/usr/lib64/libhadooppipes.a
/usr/lib64/libhadooputils.a

$ rpm -qlp hadoop-1.0.1-1.i386.rpm|grep libhdfs
/usr/lib/libhdfs.la
/usr/lib/libhdfs.so
/usr/lib/libhdfs.so.0
/usr/lib/libhdfs.so.0.0.0

$ rpm -qlp hadoop-1.0.1-1.amd64.rpm|grep libhdfs
/usr/lib64/libhdfs.la
/usr/lib64/libhdfs.so
/usr/lib64/libhdfs.so.0
/usr/lib64/libhdfs.so.0.0.0
{code}
                
> Binary tarball does not preserve platform info for native builds, and RPMs fail to provide needed symlinks for libhadoop.so
> ---------------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-8037
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8037
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: build
>    Affects Versions: 1.0.1
>            Reporter: Matt Foley
>            Assignee: Giridharan Kesavan
>            Priority: Blocker
>         Attachments: hadoop-8027-1.patch, hadoop-8037-1.patch, hadoop-8037-2.patch, hadoop-8037.patch
>
>
> The source tarball uses "package" ant target, which includes both sets of native builds (32 and 64 bit libraries), under subdirectories that are named for the supported platform, so you can tell what they are.
> The binary tarball uses the "bin-package" ant target, which projects both sets of native builds into a single directory, stripping out the platform names from the directory paths.  Since the native built libraries have identical names, only one of each survives the process.  Afterward, there is no way to know whether they are intended for 32 or 64 bit environments.
> It seems to be done this way as a step toward building the rpm and deb artifacts.  But the rpms and debs are self-identifying as to the platform they were built for, and contain only one set of libs each, while the binary tarball isn't.  The binary tarball should have the same platform-specific subdirectories that the full tarball does; but this means that the rpm and deb builds have to be more careful about include/exclude specs for what goes into those artifacts.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira