hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brian Bockelman <bbock...@cse.unl.edu>
Subject Re: libhdfs install dep
Date Tue, 25 Sep 2012 16:14:11 GMT
Hi Rodrigo,

The hadoop RPMs are a bit deficient compared to those you would find from your Linux distribution.

For example, look at the Apache RPM you used:

[bbockelm@rcf-bockelman ~]$ rpm -qp http://mirrors.sonic.net/apache/hadoop/common/hadoop-1.0.3/hadoop-1.0.3-1.x86_64.rpm
hadoop = 1.0.3-1

Normally, you would expect to see something like this (using the CDH4 distribution as an example)
as it contains a shared library:

[bbockelm@brian-test ~]$ rpm -q --provides hadoop-libhdfs
hadoop-libhdfs = 2.0.0+88-1.cdh4.0.0.p0.30.osg.el5
hadoop-libhdfs = 2.0.0+88-1.cdh4.0.0.p0.30.osg.el5

Because the Apache RPM does not list itself as providing libhdfs.so.0()(64bit), it breaks
your automatic RPM dependency detection.

[Aside: I know from experience that building a high-quality (as in, follows the Fedora Packaging
Guidelines) RPM for Java software is incredibly hard as the packaging approaches between the
Linux distributions and Java community are incredibly divergent.  Not to say that the Java
approach is inherently wrong, it's just different, and does not translate naturally to RPM.
 Accordingly, to take Hadoop and make a rule-abiding RPM in Fedora would be hundreds of hours
of work.  It's one of those things that appear to be much easier than it is to accomplish.]

The Hadoop community is very friendly, and I'm sure they would accept any patches to fix this
oversight in future releases.


On Sep 25, 2012, at 7:57 AM, "Pastrana, Rodrigo (RIS-BCT)" <Rodrigo.Pastrana@lexisnexis.com>

> Leo, yes I'm working with hadoop-1.0.1-1.amd64.rpm from Apache's download site.
> The rpm installs libhdfs in /usr/lib64 so I'm not sure why I would need the hadoop-<*>libhdfs*
> Any idea why the installed /usr/lib64/libhdfs.so is not detected by the package managers?
> Thanks, Rodrigo.
> -----Original Message-----
> From: Leo Leung [mailto:lleung@ddn.com] 
> Sent: Tuesday, September 25, 2012 2:11 AM
> To: common-user@hadoop.apache.org
> Subject: RE: libhdfs install dep
> Rodrigo,
>  Assuming you are asking for hadoop 1.x
>  You are missing the hadoop-<*>libhdfs* rpm.
>  Build it or get it from the vendor you got your hadoop from.
> -----Original Message-----
> From: Pastrana, Rodrigo (RIS-BCT) [mailto:Rodrigo.Pastrana@lexisnexis.com] 
> Sent: Monday, September 24, 2012 8:20 PM
> To: 'core-user@hadoop.apache.org'
> Subject: libhdfs install dep
> Anybody know why libhdfs.so is not found by package managers on CentOS 64 and OpenSuse64?

> I hava an rpm which declares Hadoop as a dependacy, but the package managers (KPackageKit,
zypper, etc) report libhdfs.so as a missing dependency eventhough Hadoop has been installed
via rpm package, and libhdfs.so is installed as well. 
> Thanks, Rodrigo.
> -----------------------------------------
> The information contained in this e-mail message is intended only
> for the personal and confidential use of the recipient(s) named
> above. This message may be an attorney-client communication and/or
> work product and as such is privileged and confidential. If the
> reader of this message is not the intended recipient or an agent
> responsible for delivering it to the intended recipient, you are
> hereby notified that you have received this document in error and
> that any review, dissemination, distribution, or copying of this
> message is strictly prohibited. If you have received this
> communication in error, please notify us immediately by e-mail, and
> delete the original message.

View raw message