Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Date: Wed, 8 Feb 2012 10:27:01 +0000 (UTC)
From: "Matt Foley (Issue Comment Edited) (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Message-ID: 
 <1727656886.14105.1328696821283.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <825613320.13754.1328689919661.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Issue Comment Edited] (HADOOP-8037) Binary tarball does not
 preserve platform info for native builds, and fails to provide needed
 symlinks for libhadoop.so
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-8037?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13203442#comment-13203442 ] 

Matt Foley edited comment on HADOOP-8037 at 2/8/12 10:26 AM:
-------------------------------------------------------------

The symlinks are now good!  However, the rpms are retaining some paths they shouldn't: {code}
$ rpm -qlp hadoop-1.0.1-1.i386.rpm|grep libhadoop
/usr/c++/Linux-i386-32/lib/libhadooppipes.a
/usr/c++/Linux-i386-32/lib/libhadooputils.a
/usr/lib/libhadoop.a
/usr/lib/libhadoop.la
/usr/lib/libhadoop.so
/usr/lib/libhadoop.so.1
/usr/lib/libhadoop.so.1.0.0
/usr/lib/libhadooppipes.a
/usr/lib/libhadooputils.a

$ rpm -qlp hadoop-1.0.1-1.amd64.rpm|grep libhadoop
/usr/c++/Linux-amd64-64/lib/libhadooppipes.a
/usr/c++/Linux-amd64-64/lib/libhadooputils.a
/usr/c++/Linux-i386-32/lib/libhadooppipes.a
/usr/c++/Linux-i386-32/lib/libhadooputils.a
/usr/lib64/libhadoop.a
/usr/lib64/libhadoop.la
/usr/lib64/libhadoop.so
/usr/lib64/libhadoop.so.1
/usr/lib64/libhadoop.so.1.0.0
/usr/lib64/libhadooppipes.a
/usr/lib64/libhadooputils.a
{code}
I think the /usr/c++/<platform>/lib paths shouldn't be in the rpms, only in the bin-tarball.

Also, in the bin-tarball, the libhadooppipes and utils libs are handled inconsistently: {code}
$ tar ztf hadoop-1.0.1-bin.tar.gz|grep libhadoop
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhadooppipes.a
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhadooputils.a
hadoop-1.0.1/c++/Linux-i386-32/lib/libhadooppipes.a
hadoop-1.0.1/c++/Linux-i386-32/lib/libhadooputils.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.la
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1.0.0
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadooppipes.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadooputils.a
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.a
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.la
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1.0.0
{code}
I think the files {code}
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadooppipes.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadooputils.a
{code}
shouldn't be there.

The source tarball looked fine.

Finally, please remove the commented-out line in build.xml: {code} 
<!--<delete dir="${dist.dir}"/> --> {code}

Thanks.

                
      was (Author: mattf):
    The symlinks are now good!  However, the rpms are retaining some paths they shouldn't: {code}
$ rpm -qlp hadoop-1.0.1-1.i386.rpm|grep libhadoop
/usr/c++/Linux-i386-32/lib/libhadooppipes.a
/usr/c++/Linux-i386-32/lib/libhadooputils.a
/usr/lib/libhadoop.a
/usr/lib/libhadoop.la
/usr/lib/libhadoop.so
/usr/lib/libhadoop.so.1
/usr/lib/libhadoop.so.1.0.0
/usr/lib/libhadooppipes.a
/usr/lib/libhadooputils.a

$ rpm -qlp hadoop-1.0.1-1.amd64.rpm|grep libhadoop
/usr/c++/Linux-amd64-64/lib/libhadooppipes.a
/usr/c++/Linux-amd64-64/lib/libhadooputils.a
/usr/c++/Linux-i386-32/lib/libhadooppipes.a
/usr/c++/Linux-i386-32/lib/libhadooputils.a
/usr/lib64/libhadoop.a
/usr/lib64/libhadoop.la
/usr/lib64/libhadoop.so
/usr/lib64/libhadoop.so.1
/usr/lib64/libhadoop.so.1.0.0
/usr/lib64/libhadooppipes.a
/usr/lib64/libhadooputils.a
{code}
I think the /usr/c++/<platform>/lib paths shouldn't be in the rpms, only in the bin-tarball.

Also, in the bin-tarball, the libhadooppipes and utils libs are handled inconsistently: {code}
$ tar ztf hadoop-1.0.1-bin.tar.gz|grep libhadoop
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhadooppipes.a
hadoop-1.0.1/c++/Linux-amd64-64/lib/libhadooputils.a
hadoop-1.0.1/c++/Linux-i386-32/lib/libhadooppipes.a
hadoop-1.0.1/c++/Linux-i386-32/lib/libhadooputils.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.la
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadoop.so.1.0.0
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadooppipes.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadooputils.a
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.a
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.la
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1
hadoop-1.0.1/lib/native/Linux-i386-32/libhadoop.so.1.0.0

I think the files {code}
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadooppipes.a
hadoop-1.0.1/lib/native/Linux-amd64-64/libhadooputils.a
{code}
shouldn't be there.

The source tarball looked fine.

Finally, please remove the commented-out line in build.xml: {code} 
<!--<delete dir="${dist.dir}"/> --> {code}

Thanks.

                  
> Binary tarball does not preserve platform info for native builds, and fails to provide needed symlinks for libhadoop.so
> -----------------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-8037
>                 URL: https://issues.apache.org/jira/browse/HADOOP-8037
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: build
>    Affects Versions: 1.0.1
>            Reporter: Matt Foley
>            Assignee: Giridharan Kesavan
>         Attachments: hadoop-8037.patch
>
>
> The source tarball uses "package" ant target, which includes both sets of native builds (32 and 64 bit libraries), under subdirectories that are named for the supported platform, so you can tell what they are.
> The binary tarball uses the "bin-package" ant target, which projects both sets of native builds into a single directory, stripping out the platform names from the directory paths.  Since the native built libraries have identical names, only one of each survives the process.  Afterward, there is no way to know whether they are intended for 32 or 64 bit environments.
> It seems to be done this way as a step toward building the rpm and deb artifacts.  But the rpms and debs are self-identifying as to the platform they were built for, and contain only one set of libs each, while the binary tarball isn't.  The binary tarball should have the same platform-specific subdirectories that the full tarball does; but this means that the rpm and deb builds have to be more careful about include/exclude specs for what goes into those artifacts.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira