hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Vladislav Feigin <vladi...@gmail.com>
Subject Re: 答复: TaskTracker Error
Date Thu, 23 Feb 2012 11:57:33 GMT
Hi
Check also passwordless SSH is configured properly between the nodes.
Vladi

נשלח מה-iPad שלי

ב-23 Feb 2012, בשעה 12:10, "tgh" <guanhua.tian@ia.ac.cn> כתב/ה:

> Hi
>    I use ubuntu , the firewall seems off , for three virtual machine, and how to solve
ERROR ,  cloud you help me ?
> 
> root@ubuntu:/home/hadoop-0.20.2# service iptables status
> iptables: unrecognized service
> 
> root@ubuntu:/home/hadoop-0.20.2# ufw disable
> Firewall stopped and disabled on system startup
> root@ubuntu:/home/hadoop-0.20.2# ufw status
> Status: inactive
> root@ubuntu:/home/hadoop-0.20.2#
> 
> 
> this is port by Java on master 192.168.164.128
> root@ubuntu:~# 
> root@ubuntu:~# netstat -nap|grep java
> tcp6       0      0 :::41095                :::*                    LISTEN      4000/java
      
> tcp6       0      0 :::50090                :::*                    LISTEN      4222/java
      
> tcp6       0      0 :::50060                :::*                    LISTEN      4492/java
      
> tcp6       0      0 :::42316                :::*                    LISTEN      4297/java
      
> tcp6       0      0 192.168.164.136:9100    :::*                    LISTEN      3800/java
      
> tcp6       0      0 192.168.164.136:9101    :::*                    LISTEN      4297/java
      
> tcp6       0      0 :::50030                :::*                    LISTEN      4297/java
      
> tcp6       0      0 :::33297                :::*                    LISTEN      3800/java
      
> tcp6       0      0 127.0.0.1:60722         :::*                    LISTEN      4492/java
      
> tcp6       0      0 :::50070                :::*                    LISTEN      3800/java
      
> tcp6       0      0 :::50010                :::*                    LISTEN      4000/java
      
> tcp6       0      0 :::50075                :::*                    LISTEN      4000/java
      
> tcp6       0      0 :::35262                :::*                    LISTEN      4222/java
      
> tcp6       0      0 :::50020                :::*                    LISTEN      4000/java
      
> tcp6       0      0 192.168.164.136:58531   192.168.164.136:9101    ESTABLISHED 4492/java
      
> tcp6       0      0 192.168.164.136:9100    192.168.164.136:37493   ESTABLISHED 3800/java
      
> tcp6       0      0 192.168.164.136:37490   192.168.164.136:9100    ESTABLISHED 4297/java
      
> tcp6       0      0 192.168.164.136:9100    192.168.164.137:53796   ESTABLISHED 3800/java
      
> tcp6       0      0 192.168.164.136:9100    192.168.164.136:37490   ESTABLISHED 3800/java
      
> tcp6       0      0 192.168.164.136:9100    192.168.164.138:40077   ESTABLISHED 3800/java
      
> tcp6       0      0 192.168.164.136:37493   192.168.164.136:9100    ESTABLISHED 4000/java
      
> unix  2      [ ]         STREAM     CONNECTED     21015    4492/java           
> unix  2      [ ]         STREAM     CONNECTED     20907    4297/java           
> unix  2      [ ]         STREAM     CONNECTED     20204    4222/java           
> unix  2      [ ]         STREAM     CONNECTED     19574    4000/java           
> unix  2      [ ]         STREAM     CONNECTED     19293    3800/java           
> root@ubuntu:~#
> 
> this is on slaves 192.168.164.137
> root@ubuntu:/home/hadoop-0.20.2# 
> root@ubuntu:/home/hadoop-0.20.2# netstat -nap|grep java
> tcp6       0      0 :::50060                :::*                    LISTEN      13130/java
     
> tcp6       0      0 127.0.0.1:40112         :::*                    LISTEN      13130/java
     
> tcp6       0      0 :::35703                :::*                    LISTEN      12949/java
     
> tcp6       0      0 :::50010                :::*                    LISTEN      12949/java
     
> tcp6       0      0 :::50075                :::*                    LISTEN      12949/java
     
> tcp6       0      0 :::50020                :::*                    LISTEN      12949/java
     
> tcp6       0      0 192.168.164.137:53796   192.168.164.136:9100    ESTABLISHED 12949/java
     
> tcp6       0      0 192.168.164.137:43216   192.168.164.136:9101    ESTABLISHED 13130/java
     
> unix  2      [ ]         STREAM     CONNECTED     51464    13130/java          
> unix  2      [ ]         STREAM     CONNECTED     49229    12949/java          
> root@ubuntu:/home/hadoop-0.20.2#
> 
> 
> 
> 
> 
> -----邮件原件-----
> 发件人: common-user-return-32874-guanhua.tian=ia.ac.cn@hadoop.apache.org [mailto:common-user-return-32874-guanhua.tian=ia.ac.cn@hadoop.apache.org]
代表 Harsh J
> 发送时间: 2012年2月23日 17:31
> 收件人: common-user@hadoop.apache.org
> 主题: Re: TaskTracker Error
> 
> Have you ensured your firewall is off on all instances, or appropriately configured if
you need them?
> 
> $ service iptables stop
> 
> It is turned on by default on most distributions. I know CentOS6 turns it on by default,
with some rules.
> 
> On Thu, Feb 23, 2012 at 2:33 PM, tgh <guanhua.tian@ia.ac.cn> wrote:
>> Hi
>> 
>>        I setup hadoop with hadoop 0.20.2
>> 
>> 
>> 
>>        I use three virtual machines on vmware,
>> 
>>        The three virtual machine could ssh with each other,
>> 
>> ERROR rise ,   the tasktracker on slave 192.168.164.137 and 
>> 192.168.164.138 cloud not connect to master, while the tasktracker on 
>> 192.168.164.136 seems no error,
>> 
>> 
>> 
>> Cloud you help me
>> 
>> 
>> 
>> The conf file is set as follows,
>> 
>> root@ubuntu:/home/hadoop-0.20.2/conf# cat masters
>> 
>> 192.168.164.136
>> 
>> root@ubuntu:/home/hadoop-0.20.2/conf# cat slaves
>> 
>> 192.168.164.136
>> 
>> 192.168.164.137
>> 
>> 192.168.164.138
>> 
>> root@ubuntu:/home/hadoop-0.20.2/conf#
>> 
>> root@ubuntu:/home/hadoop-0.20.2/conf# cat core-site.xml
>> 
>> <?xml version="1.0"?>
>> 
>> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>> 
>> 
>> 
>> <!-- Put site-specific property overrides in this file. -->
>> 
>> 
>> 
>> <configuration>
>> 
>> <property>
>> 
>>   <name>fs.default.name</name>
>> 
>>   <value>hdfs://192.168.164.136:9100</value>
>> 
>> </property>
>> 
>> <property>
>> 
>>   <name>hadoop.tmp.dir</name>
>> 
>>   <value>/home/hadoop-0.20.2/tmp/</value>
>> 
>> </property>
>> 
>> <property>
>> 
>>   <name>dfs.replication</name>
>> 
>>   <value>1</value>
>> 
>> </property>
>> 
>> <!-- property>
>> 
>>   <name>mapred.child.java.opts</name>
>> 
>>   <value>-Xmx128m</value>
>> 
>> </property>
>> 
>> <property>
>> 
>>   <name>dfs.block.size</name>
>> 
>>   <value>5120000</value>
>> 
>>   <description>The default block size for new files.</description>
>> 
>> </property -->
>> 
>> </configuration>
>> 
>> 
>> 
>> root@ubuntu:/home/hadoop-0.20.2/conf# cat mapred-site.xml
>> 
>> <?xml version="1.0"?>
>> 
>> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
>> 
>> 
>> 
>> <!-- Put site-specific property overrides in this file. -->
>> 
>> 
>> 
>> <configuration>
>> 
>> <property>
>> 
>>   <name>mapred.job.tracker</name>
>> 
>>   <value>192.168.164.136:9101</value>
>> 
>> </property>
>> 
>> </configuration>
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> now ERROR rise ,   the tasktracker on slave 192.168.164.137 and
>> 192.168.164.138 cloud not connect to master, while the tasktracker on
>> 192.168.164.136 seems no error,
>> 
>> 
>> 
>> this is the log on 192.168.164.138,
>> 
>> root@ubuntu:/home/hadoop-0.20.2/logs#
>> 
>> root@ubuntu:/home/hadoop-0.20.2/logs# cat 
>> hadoop-root-tasktracker-ubuntu.log
>> 
>> 
>> 2012-02-23 00:44:10,851 INFO org.apache.hadoop.mapred.TaskTracker:
>> STARTUP_MSG:
>> 
>> /************************************************************
>> 
>> STARTUP_MSG: Starting TaskTracker
>> 
>> STARTUP_MSG:   host = ubuntu/127.0.1.1
>> 
>> STARTUP_MSG:   args = []
>> 
>> STARTUP_MSG:   version = 0.20.2
>> 
>> STARTUP_MSG:   build =
>> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-0.20 -r 
>> 911707; compiled by 'chrisdo' on Fri Feb 19 08:07:34 UTC 2010
>> 
>> ************************************************************/
>> 
>> 2012-02-23 00:44:16,080 INFO org.mortbay.log: Logging to
>> org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via 
>> org.mortbay.log.Slf4jLog
>> 
>> 2012-02-23 00:44:16,199 INFO org.apache.hadoop.http.HttpServer: Port 
>> returned by webServer.getConnectors()[0].getLocalPort() before open() is -1.
>> Opening the listener on 50060
>> 
>> 2012-02-23 00:44:16,205 INFO org.apache.hadoop.http.HttpServer:
>> listener.getLocalPort() returned 50060
>> webServer.getConnectors()[0].getLocalPort() returned 50060
>> 
>> 2012-02-23 00:44:16,205 INFO org.apache.hadoop.http.HttpServer: Jetty 
>> bound to port 50060
>> 
>> 2012-02-23 00:44:16,205 INFO org.mortbay.log: jetty-6.1.14
>> 
>> 2012-02-23 00:45:08,741 INFO org.mortbay.log: Started
>> SelectChannelConnector@0.0.0.0:50060
>> 
>> 2012-02-23 00:45:08,808 INFO org.apache.hadoop.metrics.jvm.JvmMetrics:
>> Initializing JVM Metrics with processName=TaskTracker, sessionId=
>> 
>> 2012-02-23 00:45:08,848 INFO org.apache.hadoop.ipc.metrics.RpcMetrics:
>> Initializing RPC Metrics with hostName=TaskTracker, port=49689
>> 
>> 2012-02-23 00:45:08,909 INFO org.apache.hadoop.ipc.Server: IPC Server
>> Responder: starting
>> 
>> 2012-02-23 00:45:08,912 INFO org.apache.hadoop.mapred.TaskTracker:
>> TaskTracker up at: localhost/127.0.0.1:49689
>> 
>> 2012-02-23 00:45:08,912 INFO org.apache.hadoop.mapred.TaskTracker: 
>> Starting tracker tracker_ubuntu:localhost/127.0.0.1:49689
>> 
>> 2012-02-23 00:45:08,911 INFO org.apache.hadoop.ipc.Server: IPC Server 
>> listener on 49689: starting
>> 
>> 2012-02-23 00:45:08,911 INFO org.apache.hadoop.ipc.Server: IPC Server 
>> handler 0 on 49689: starting
>> 
>> 2012-02-23 00:45:08,911 INFO org.apache.hadoop.ipc.Server: IPC Server 
>> handler 1 on 49689: starting
>> 
>> 2012-02-23 00:45:08,911 INFO org.apache.hadoop.ipc.Server: IPC Server 
>> handler 2 on 49689: starting
>> 
>> 2012-02-23 00:45:08,919 INFO org.apache.hadoop.ipc.Server: IPC Server 
>> handler 3 on 49689: starting
>> 
>> 2012-02-23 00:47:53,638 INFO org.apache.hadoop.mapred.TaskTracker:  
>> Using MemoryCalculatorPlugin :
>> org.apache.hadoop.util.LinuxMemoryCalculatorPlugin@cafb56
>> 
>> 2012-02-23 00:47:53,641 INFO org.apache.hadoop.mapred.TaskTracker: 
>> Starting
>> thread: Map-events fetcher for all reduce tasks on
>> tracker_ubuntu:localhost/127.0.0.1:49689
>> 
>> 2012-02-23 00:47:53,646 WARN org.apache.hadoop.mapred.TaskTracker:
>> TaskTracker's totalMemoryAllottedForTasks is -1. TaskMemoryManager is 
>> disabled.
>> 
>> 2012-02-23 00:47:53,647 INFO org.apache.hadoop.mapred.IndexCache: 
>> IndexCache created with max memory = 10485760
>> 
>> 2012-02-23 00:47:55,110 INFO org.apache.hadoop.ipc.Client: Retrying 
>> connect to server: ubuntu.local/192.168.164.138:9100. Already tried 0 time(s).
>> 
>> 2012-02-23 00:47:56,112 INFO org.apache.hadoop.ipc.Client: Retrying 
>> connect to server: ubuntu.local/192.168.164.138:9100. Already tried 1 time(s).
>> 
>> 2012-02-23 00:47:57,114 INFO org.apache.hadoop.ipc.Client: Retrying 
>> connect to server: ubuntu.local/192.168.164.138:9100. Already tried 2 time(s).
>> 
>> 2012-02-23 00:47:58,116 INFO org.apache.hadoop.ipc.Client: Retrying 
>> connect to server: ubuntu.local/192.168.164.138:9100. Already tried 3 time(s).
>> 
>> 2012-02-23 00:47:59,118 INFO org.apache.hadoop.ipc.Client: Retrying 
>> connect to server: ubuntu.local/192.168.164.138:9100. Already tried 4 time(s).
>> 
>> 2012-02-23 00:48:00,120 INFO org.apache.hadoop.ipc.Client: Retrying 
>> connect to server: ubuntu.local/192.168.164.138:9100. Already tried 5 time(s).
>> 
>> 2012-02-23 00:48:01,122 INFO org.apache.hadoop.ipc.Client: Retrying 
>> connect to server: ubuntu.local/192.168.164.138:9100. Already tried 6 time(s).
>> 
>> 2012-02-23 00:48:02,124 INFO org.apache.hadoop.ipc.Client: Retrying 
>> connect to server: ubuntu.local/192.168.164.138:9100. Already tried 7 time(s).
>> 
>> 2012-02-23 00:48:03,126 INFO org.apache.hadoop.ipc.Client: Retrying 
>> connect to server: ubuntu.local/192.168.164.138:9100. Already tried 8 time(s).
>> 
>> 2012-02-23 00:48:04,130 INFO org.apache.hadoop.ipc.Client: Retrying 
>> connect to server: ubuntu.local/192.168.164.138:9100. Already tried 9 time(s).
>> 
>> 2012-02-23 00:48:04,132 ERROR org.apache.hadoop.mapred.TaskTracker: 
>> Caught
>> exception: java.net.ConnectException: Call to
>> ubuntu.local/192.168.164.138:9100 failed on connection exception:
>> java.net.ConnectException: Connection refused
>> 
>>        at org.apache.hadoop.ipc.Client.wrapException(Client.java:767)
>> 
>>        at org.apache.hadoop.ipc.Client.call(Client.java:743)
>> 
>>        at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:220)
>> 
>>        at $Proxy5.getProtocolVersion(Unknown Source)
>> 
>>        at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:359)
>> 
>>        at
>> org.apache.hadoop.hdfs.DFSClient.createRPCNamenode(DFSClient.java:106)
>> 
>>        at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:207)
>> 
>>        at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:170)
>> 
>>        at
>> org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFil
>> eSyste
>> m.java:82)
>> 
>>        at
>> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:1378)
>> 
>>        at 
>> org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:66)
>> 
>>        at 
>> org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:1390)
>> 
>>        at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:196)
>> 
>>        at org.apache.hadoop.fs.Path.getFileSystem(Path.java:175)
>> 
>>        at
>> org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:103
>> 3)
>> 
>>        at 
>> org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:1720)
>> 
>>        at 
>> org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2833)
>> 
>> Caused by: java.net.ConnectException: Connection refused
>> 
>>        at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
>> 
>>        at
>> sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:592)
>> 
>>        at
>> org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.
>> java:2
>> 06)
>> 
>>        at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:404)
>> 
>>        at
>> org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:304
>> )
>> 
>>        at
>> org.apache.hadoop.ipc.Client$Connection.access$1700(Client.java:176)
>> 
>>        at org.apache.hadoop.ipc.Client.getConnection(Client.java:860)
>> 
>>        at org.apache.hadoop.ipc.Client.call(Client.java:720)
>> 
>>        ... 15 more
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
>> 
> 
> 
> 
> --
> Harsh J
> Customer Ops. Engineer
> Cloudera | http://tiny.cloudera.com/about
> 
> 

Mime
  • Unnamed multipart/alternative (inline, 7-Bit, 0 bytes)
View raw message