hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From MirrorX <mirr...@gmail.com>
Subject network configuration (etc/hosts) ?
Date Wed, 21 Dec 2011 01:21:43 GMT

dear all

i am trying for many days to get a simple hadoop cluster (with 2 nodes) to
work but i have trouble configuring the network parameters. i have properly
configured the ssh keys, and the /etc/hosts files are:

master->
127.0.0.1 localhost6.localdomain6 localhost
127.0.1.1 localhost4.localdomain4 master-pc
192.168.7.110 master
192.168.7.157 slave

slave->
127.0.1.1    localhost5.localdomain5 lab-pc
127.0.0.1    localhost3.localdomain3 localhost
192.168.7.110 master
192.168.7.157 slave

i have tried all possible combinations on the /etc/hosts files but i still
cannot make it work. i either get errors 'too many fetch failures' and by
examining the logs of the slave i see this 
' INFO org.apache.hadoop.mapred.TaskTracker:
attempt_201112210259_0002_r_000000_0 0.11111112% reduce > copy (1 of 3 at
0.03MB/s)'

or i get errors like -> 
'INFO mapred.JobClient: Task Id : attempt_201112210308_0001_r_000000_0,
Status : FAILED
Shuffle Error: Exceeded MAX_FAILED_UNIQUE_FETCHES; bailing-out.'

i have searched into many similar posts on the web but i still cannot find
the solution. could you please help me?

when i run the same job only on the master it is completed fine, and i can
connect via ssh from every node to every node and from each node to itself,
that's why i think there is something wrong with the network configuration

thank you in advance for your help



-- 
View this message in context: http://old.nabble.com/network-configuration-%28etc-hosts%29---tp33013719p33013719.html
Sent from the Hadoop core-user mailing list archive at Nabble.com.


Mime
View raw message