hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From hadoop user <using.had...@gmail.com>
Subject Re: task tracker fails to start in one of the slave node
Date Sat, 04 Jan 2014 18:54:43 GMT
answering my own question

slave1 mapred-site.xml was missing the mapred.job.tracker property and thus
taking default values for host as *local*

mapred.job.trackerlocalThe host and port that the MapReduce job tracker
runs at. If "local", then jobs are run in-process as a single map and
reduce task.

<configuration>
<property>
<name>mapred.job.tracker</name>
<value>*hdfs://ec2-namdenode(master).*compute.amazonaws.com:8021</value>
</property>
</configuration>.


On Sat, Jan 4, 2014 at 12:34 PM, hadoop user <using.hadoop@gmail.com> wrote:

> I am trying to setup 4 node cluster on ec2
>
> ec2 machine setup is as follow
>
> 1 namenode, (master) 1 secondary namenode , and 2 slave nodes
>
> after issuing start-all.sh on master , all daemons starts as expected with
> only one issue
>
> on slave2 - data node and tasktracker starts , but on slave1 only datanode
> starts and tasktracker fails with below exception
>
> my mapred-site.xml is straight forward
>
> <configuration>
> <property>
> <name>mapred.job.tracker</name>
> <value>*hdfs://ec2-namdenode(master).*compute.amazonaws.com:8021</value>
> </property>
> </configuration>
>
> what I am doing wrong here?
>
> *log output*
>
>
> 2014-01-04 16:58:48,384 INFO org.apache.hadoop.mapred.TaskTracker:
> STARTUP_MSG:
> /************************************************************
> STARTUP_MSG: Starting TaskTracker
> STARTUP_MSG:   host = ec2-public-hostname.compute.amazonaws.com/
> <ec2-public-ip>
> STARTUP_MSG:   args = []
> STARTUP_MSG:   version = 1.2.1
> STARTUP_MSG:   build =
> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r
> 1503152; compiled by 'mattf' on Mon Jul 22 15:23:09 PDT 2013
> STARTUP_MSG:   java = 1.7.0_45
> ************************************************************/
> 2014-01-04 16:58:49,261 INFO
> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from
> hadoop-metrics2.properties
> 2014-01-04 16:58:49,484 INFO
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
> MetricsSystem,sub=Stats registered.
> 2014-01-04 16:58:49,488 INFO
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot
> period at 10 second(s).
> 2014-01-04 16:58:49,488 INFO
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: TaskTracker metrics
> system started
> 2014-01-04 16:58:50,609 INFO org.apache.hadoop.util.NativeCodeLoader:
> Loaded the native-hadoop library
> 2014-01-04 16:58:51,005 INFO
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi
> registered.
> 2014-01-04 16:58:51,036 WARN
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already
> exists!
> *2014-01-04 16:58:51,489 ERROR org.apache.hadoop.mapred.TaskTracker: Can
> not start task tracker because java.lang.IllegalArgumentException: Does not
> contain a valid host:port authority: local*
> * at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)*
>  at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:130)
> at org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:2131)
>  at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1540)
> at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3937)
>
> 2014-01-04 16:58:51,510 INFO org.apache.hadoop.mapred.TaskTracker:
> SHUTDOWN_MSG:
> /************************************************************
> SHUTDOWN_MSG: Shutting down TaskTracker at
> ec2-public-hostname.compute.amazonaws.com/<ec2-public-ip>
> ************************************************************/
> 2014-01-04 17:02:09,229 INFO org.apache.hadoop.mapred.TaskTracker:
> STARTUP_MSG:
> /************************************************************
> STARTUP_MSG: Starting TaskTracker
> STARTUP_MSG:   host = ec2-public-hostname.compute.amazonaws.com/
> <ec2-public-ip>
> STARTUP_MSG:   args = []
> STARTUP_MSG:   version = 1.2.1
> STARTUP_MSG:   build =
> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.2 -r
> 1503152; compiled by 'mattf' on Mon Jul 22 15:23:09 PDT 2013
> STARTUP_MSG:   java = 1.7.0_45
> ************************************************************/
> 2014-01-04 17:02:09,672 INFO
> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from
> hadoop-metrics2.properties
> 2014-01-04 17:02:09,781 INFO
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
> MetricsSystem,sub=Stats registered.
> 2014-01-04 17:02:09,782 INFO
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot
> period at 10 second(s).
> 2014-01-04 17:02:09,782 INFO
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: TaskTracker metrics
> system started
> 2014-01-04 17:02:10,303 INFO org.apache.hadoop.util.NativeCodeLoader:
> Loaded the native-hadoop library
> 2014-01-04 17:02:10,494 INFO
> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi
> registered.
> 2014-01-04 17:02:10,511 WARN
> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already
> exists!
> 2014-01-04 17:02:10,744 ERROR org.apache.hadoop.mapred.TaskTracker: Can
> not start task tracker because java.lang.IllegalArgumentException: Does not
> contain a valid host:port authority: local
> at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:164)
>  at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:130)
> at org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:2131)
>  at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1540)
> at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3937)
>
> 2014-01-04 17:02:10,754 INFO org.apache.hadoop.mapred.TaskTracker:
> SHUTDOWN_MSG:
> /************************************************************
> SHUTDOWN_MSG: Shutting down TaskTracker at
> ec2-public-hostname.compute.amazonaws.com/<ec2-public-ip>
> ************************************************************/
>
>

Mime
View raw message