hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Watrous <dwmaill...@gmail.com>
Subject Re: Datanodes not connecting to the cluster
Date Thu, 24 Sep 2015 19:14:39 GMT
On one of the namenodes I have found the following warning:

2015-09-24 18:40:17,639 WARN
org.apache.hadoop.hdfs.server.datanode.DataNode: Problem connecting to
server: hadoop-master/192.168.51.4:54310

On my master node I see that the process is running and has bound that port

vagrant@hadoop-master:~/src$ sudo lsof -i :54310
COMMAND  PID   USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
java    7480 hadoop  202u  IPv4  26931      0t0  TCP hadoop-master:54310
(LISTEN)
java    7480 hadoop  212u  IPv4  28758      0t0  TCP
hadoop-master:54310->localhost:47226 (ESTABLISHED)
java    7651 hadoop  238u  IPv4  28247      0t0  TCP
localhost:47226->hadoop-master:54310 (ESTABLISHED)
hadoop@hadoop-master:~$ jps
7856 SecondaryNameNode
7651 DataNode
7480 NameNode
8106 Jps

I don't appear to have any firewall rules interfering with traffic
vagrant@hadoop-master:~/src$ sudo iptables --list
Chain INPUT (policy ACCEPT)
target     prot opt source               destination

Chain FORWARD (policy ACCEPT)
target     prot opt source               destination

Chain OUTPUT (policy ACCEPT)
target     prot opt source               destination

The iptables --list output is identical on hadoop-data1. I also show a
process attempting to connect to hadoop-master
vagrant@hadoop-data1:~$ sudo lsof -i :54310
COMMAND  PID   USER   FD   TYPE DEVICE SIZE/OFF NODE NAME
java    3823 hadoop  238u  IPv4  19304      0t0  TCP
hadoop-data1:45600->hadoop-master:54310 (SYN_SENT)

I am confused by the notation of hostname/IP:port.
All help appreciated.

Daniel



On Thu, Sep 24, 2015 at 12:51 PM, Daniel Watrous <dwmaillist@gmail.com>
wrote:

> I have a multi-node cluster with two datanodes. After running
> start-dfs.sh, I show the following processes running
>
> hadoop@hadoop-master:~$ $HADOOP_HOME/sbin/slaves.sh jps
> hadoop-master: 10933 DataNode
> hadoop-master: 10759 NameNode
> hadoop-master: 11145 SecondaryNameNode
> hadoop-master: 11567 Jps
> hadoop-data1: 5186 Jps
> hadoop-data1: 5059 DataNode
> hadoop-data2: 5180 Jps
> hadoop-data2: 5053 DataNode
>
>
> However, the other two DataNodes aren't visible.
> http://screencast.com/t/icsLnXXDk
>
> Where can I look for clues?
>

Mime
View raw message