hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chanchal James" <chanc...@gmail.com>
Subject Re: Ec2 and MR Job question
Date Mon, 16 Jun 2008 18:28:26 GMT
Hi Billy, when I tested Hadoop on an EC2 machine, I didnt come across the
hostname problem.. Probably because I changed the hostname to a public FQDN.

On Sat, Jun 14, 2008 at 10:09 PM, Billy Pearson <sales@pearsonwholesale.com>

> My second question is about the ec2 machines has anyone solved the hostname
> problem in a automated way?
> Example if I launch a ec2 server to run a task tracker the hostname
> reported back to my local cluster with its internal address
> the local reduce task can not access the map files on the ec2 machine
> because with the default hostname.
> I get a error:
> WARN org.apache.hadoop.mapred.ReduceTask: java.net.UnknownHostException:
> domU-12-31-39-00-A4-05.compute-1.internal
> <question>
> Is there a automated way to start a tasktracker on a ec2 machine with it
> useing the public hostname so the local task can get the maps from the ec2
> machines?
> example something like
> bin/hadoop-daemon.sh start tasktracker host=
> ec2-xx-xx-xx-xx.z-2.compute-1.amazonaws.com
> That I can run to start just the tasktracker with the correct hostname
> </question>
> What I am trying to do is build a custom ami image that I can just launch
> when need to add extra cpu power to my cluster and to automatically start
> the tasktracker vi a shell script that can be ran at startup.
> Billy

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message