hadoop-general mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Roman Shaposhnik <...@apache.org>
Subject Re: [VOTE] Release Apache Hadoop 0.23.1-rc2
Date Thu, 23 Feb 2012 04:48:02 GMT
On Wed, Feb 22, 2012 at 7:45 PM, Vinod Kumar Vavilapalli
<vinodkv@hortonworks.com> wrote:
> Roman,
>
> you are compiling on 64 bit machines?

I'm compiling on both 32bit and 64bit architectures.

>> not only make it impossible to build the right version
>> of container executor without hand-editing the pom.xml,
>
> The idea was to have 32 bit binary all the time

This is a flawed idea and that's one half of the reason I gave this RC
a -1. You can NOT force all 64bit environments to always have
a 32bit set of libraries, etc. Basically, with this release, all of a sudden
you force all the deployments to install quite a lot of extra RPMs/DEBs
for no other reason but to workaround a deficiency in the default Hadoop
build. Now, that would been enough to give this RC a -0 but
an added issue of prohibiting regeneration of the correct packages with the
reasonable set of binaries makes it -1.

> can you let us know what issue you are facing. During the build itself?

http://bigtop01.cloudera.org:8080/view/Bigtop-hadoop-0.23/job/Bigtop-hadoop-0.23-Hadoop/12/

Click on any of the distributions to see how the build fails.

Note -- these are clean VMs. We take special precautions
not to pollute them with any extra dependencies. These VMs are
what distribution providers (CentOS, SLES, Canonical) have deemed
to be the worthy of public consumption on Amazon's EC2.

>> but also make it impossible to deploy the secure
>> version of hadoop without additional burden of installing
>> 32bit legacy support on all the slave nodes.
>
> Again, what do you mean by this? You mean you will have to install all
> 32 bit dependencies that the container-executor may have on all nodes?

That is correct. These dependencies are not available by default, have
an extra layer of complexity and are being slowly phased out of support
(to the best of my knowledge).

> AFAIU, if you have a single 32bit lib/binary on a 64 bit machine, you
> don't need any other 32bit lib dependencies. Please correct me if I am
> wrong.

$ tar xzvf hadoop-0.23.1.tag.gz
$ ldd bin/container-executor
	linux-gate.so.1 =>  (0xf770a000)
	libc.so.6 => /lib32/libc.so.6 (0xf758d000)
	/lib/ld-linux.so.2 (0xf770b000)

That /lib32/libc.so.6 dependency is a problem.

> Myself cannot test it exactly like the way you are testing as I
> don't have access to nodes which don't have 32bits installed, so more
> specific details are helpful.

You don't have to go far to get a testbed -- any default 64bit VM
of all the major Linux distributions on EC2 would suffice.

Thanks,
Roman.

Mime
View raw message