hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Harsh J <ha...@cloudera.com>
Subject Re: No 32-bit taskcontroller on Hadoop 1.0.3
Date Tue, 18 Sep 2012 03:38:54 GMT
Hey Yongzhi,

You'll need to build it manually on your 32-bit machine. Run the build

(Depending on your version you may or may not require jdk5, we had
removed that dependency recently. The rest of the tools mentioned, you
should require for a full tar generation.)

ant -Dcompile.native=true -Dcompile.c++=true -Dlibhdfs=true
-Dlibrecordio=true -Dhadoop.conf.dir=/etc/hadoop
-Dfindbugs.home=$HOME/tools/findbugs/latest veryclean clean
create-c++-configure task-controller tar

This should build a fully formed tarball that should have 32-bit
natives in it. Use this tarball to deploy your cluster.

Or if you just want the task-controller and the natives for your arch,
do (Shouldn't require any tool dependencies):

ant -Dcompile.native=true -Dcompile.c++=true -Dlibhdfs=true
-Dlibrecordio=true veryclean clean compile create-c++-configure

Alternatively, I encourage looking at using packages for your arch,
and at the Apache Bigtop (incubating) project:

On Tue, Sep 18, 2012 at 8:34 AM, Yongzhi Wang
<wang.yongzhi2009@gmail.com> wrote:
> Dear All,
> I am currently deploying hadoop 1.0.3 on my Debian 32-bit Linux. I
> think need a 32-bit binary file taskcontroller. However, I found the
> binary
> files provided in hadoop 1.0.3 is 64 bit. I downloaded the hadoop
> build file from server jenkins
> (https://builds.apache.org/job/Hadoop-1.0-Build/ws/trunk/build/c++-build/Linux-i386-32/task-controller/).
> It's still a 64 bit file.
> I got the following errors when I start task tracker using the hadoop
> 64-bit taskcontroller:
> 12/09/17 11:59:58 ERROR mapred.TaskTracker: Can not start task tracker
> because java.io.IOException: Task controller setup failed because of
> invalidpermissions/ownership with exit code 126
>         at org.apache.hadoop.mapred.LinuxTaskController.setup(LinuxTaskController.java:143)
>         at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1452)
>         at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3742)
> Caused by: org.apache.hadoop.util.Shell$ExitCodeException:
> /opt/ywang/hadoop-1.0.3/libexec/../bin/task-controller:
> /opt/ywang/hadoop-1.0.3/libexec/../bin/task-controller: cannot execute
> binary file
>         at org.apache.hadoop.util.Shell.runCommand(Shell.java:255)
>         at org.apache.hadoop.util.Shell.run(Shell.java:182)
>         at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:375)
>         at org.apache.hadoop.mapred.LinuxTaskController.setup(LinuxTaskController.java:137)
> I am wondering if not providing 32-bit of taskcontroller is a build
> bug, or 64-bit taskcontroller can be used somehow on the 32-bit
> platform? If no 32-bit executable is provided in the daily build of
> hadoop, how can I build one by myself?
> Thanks!

Harsh J

View raw message