hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Konstantin Boudnik <...@apache.org>
Subject Re: Starting JobTracker Locally but binding to remote Address
Date Tue, 31 May 2011 21:59:18 GMT
On Tue, May 31, 2011 at 06:21PM, gordoslocos wrote:
> Eeeeeeeeh why? Isnt That the config for the jobtracker? Slave2 has been defined in my
/etc/hosts files.
> Should those lines not be in both nodes?

Indeed, but you are running MR start script on slave3 meaning that JT will be
started on slave3 whatever the configuration says: start-mapred.sh isn't that
smart and doesn't check your configs.

Cos

> Thanks for helping!
> Pony
> 
> On 31/05/2011, at 18:12, Konstantin Boudnik <cos@apache.org> wrote:
> 
> > This seems to be your problem, really...
> > * <name>mapred.job.tracker</name>*
> > * <value>slave2:9001</value>*
> > 
> > On Tue, May 31, 2011 at 06:07PM, Juan P. wrote:
> >> Hi Guys,
> >> I recently configured my cluster to have 2 VMs. I configured 1
> >> machine (slave3) to be the namenode and another to be the
> >> jobtracker (slave2). They both work as datanode/tasktracker as well.
> >> 
> >> Both configs have the following contents in their masters and slaves file:
> >> *slave2*
> >> *slave3*
> >> 
> >> Both machines have the following contents on their mapred-site.xml file:
> >> *<?xml version="1.0"?>*
> >> *<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>*
> >> *
> >> *
> >> *<!-- Put site-specific property overrides in this file. -->*
> >> *
> >> *
> >> *<configuration>*
> >> * <property>*
> >> * <name>mapred.job.tracker</name>*
> >> * <value>slave2:9001</value>*
> >> * </property>*
> >> *</configuration>*
> >> 
> >> Both machines have the following contents on their core-site.xml file:
> >> *<?xml version="1.0"?>*
> >> *<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>*
> >> *
> >> *
> >> *<!-- Put site-specific property overrides in this file. -->*
> >> *
> >> *
> >> *<configuration>*
> >> * <property>*
> >> * <name>fs.default.name</name>*
> >> * <value>hdfs://slave3:9000</value>*
> >> * </property>*
> >> *</configuration>*
> >> 
> >> When I log into the namenode and I run the start-all.sh script, everything
> >> but the jobtracker starts. In the log files I get the following exception:
> >> 
> >> */*************************************************************
> >> *STARTUP_MSG: Starting JobTracker*
> >> *STARTUP_MSG:   host = slave3/10.20.11.112*
> >> *STARTUP_MSG:   args = []*
> >> *STARTUP_MSG:   version = 0.20.2*
> >> *STARTUP_MSG:   build =
> >> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r
> >> 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010*
> >> *************************************************************/*
> >> *2011-05-31 13:54:06,940 INFO org.apache.hadoop.mapred.JobTracker: Scheduler
> >> configured with (memSizeForMapSlotOnJT, memSizeForReduceSlotOnJT,
> >> limitMaxMemForMapTasks, limitMaxMemForReduceTasks) (-1, -1, -1, -1)*
> >> *2011-05-31 13:54:07,086 FATAL org.apache.hadoop.mapred.JobTracker:
> >> java.net.BindException: Problem binding to slave2/10.20.11.166:9001 : Cannot
> >> assign requested address*
> >> *        at org.apache.hadoop.ipc.Server.bind(Server.java:190)*
> >> *        at org.apache.hadoop.ipc.Server$Listener.<init>(Server.java:253)*
> >> *        at org.apache.hadoop.ipc.Server.<init>(Server.java:1026)*
> >> *        at org.apache.hadoop.ipc.RPC$Server.<init>(RPC.java:488)*
> >> *        at org.apache.hadoop.ipc.RPC.getServer(RPC.java:450)*
> >> *        at org.apache.hadoop.mapred.JobTracker.<init>(JobTracker.java:1595)
> >> *
> >> *        at
> >> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:183)*
> >> *        at
> >> org.apache.hadoop.mapred.JobTracker.startTracker(JobTracker.java:175)*
> >> *        at org.apache.hadoop.mapred.JobTracker.main(JobTracker.java:3702)*
> >> *Caused by: java.net.BindException: Cannot assign requested address*
> >> *        at sun.nio.ch.Net.bind(Native Method)*
> >> *        at
> >> sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:126)*
> >> *        at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:59)
> >> *
> >> *        at org.apache.hadoop.ipc.Server.bind(Server.java:188)*
> >> *        ... 8 more*
> >> *
> >> *
> >> *2011-05-31 13:54:07,096 INFO org.apache.hadoop.mapred.JobTracker:
> >> SHUTDOWN_MSG:*
> >> */*************************************************************
> >> *SHUTDOWN_MSG: Shutting down JobTracker at slave3/10.20.11.112*
> >> *************************************************************/*
> >> 
> >> 
> >> As I see it, from the lines
> >> 
> >> *STARTUP_MSG: Starting JobTracker*
> >> *STARTUP_MSG:   host = slave3/10.20.11.112*
> >> 
> >> the namenode (slave3) is trying to run the jobtracker locally but when it
> >> starts the jobtracker server it binds it to the slave2 address and of course
> >> fails:
> >> 
> >> *Problem binding to slave2/10.20.11.166:9001*
> >> 
> >> What do you guys think could be going wrong?
> >> 
> >> Thanks!
> >> Pony

Mime
View raw message