hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fei Hu <hufe...@gmail.com>
Subject Hadoop Installation on Multihomed Networks
Date Tue, 11 Nov 2014 15:47:04 GMT
Hi,

I am trying to install Hadoop1.0.4 on multihomed networks. I have done it as the link http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsMultihoming.html
<http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsMultihoming.html>.
But it still doesn’t work.

The datanode could not work. Its log is as the following. In my hdfs-site.xml, I set the ip:10.50.0.10
for datanode, but in the following report, host = yngcr10nc01/10.10.0.10.I think it may be
because in /etc/hosts file, I add the pair of ip and hostname:10.10.0.10 YNGCR10NC01 before.


The problem now is that I could not add 10.50.0.10 YNGCR10NC01 into hosts file, because 10.10.0.10
YNGCR10NC01 is necessary for another program.

Is there any way to solve the problem on multihomed networks?

Thanks in advance,
Fei Hu





2014-11-11 04:20:28,228 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG:

/************************************************************
STARTUP_MSG: Starting DataNode
STARTUP_MSG:   host = yngcr10nc01/10.10.0.10
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 1.0.4
STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0
-r 1393290; compiled by 'hortonfo' on Wed Oct  3 05:13:58 UTC 2012
************************************************************/
2014-11-11 04:20:28,436 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties
from hadoop-metrics2.properties
2014-11-11 04:20:28,446 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for
source MetricsSystem,sub=Stats registered.
2014-11-11 04:20:28,447 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled
snapshot period at 10 second(s).
2014-11-11 04:20:28,447 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode metrics
system started
2014-11-11 04:20:28,572 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for
source ugi registered.
2014-11-11 04:20:28,607 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop
library
2014-11-11 04:20:29,870 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: yngcr11hm01/10.50.0.5:9000.
Already tried 0 time(s).
2014-11-11 04:20:30,871 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: yngcr11hm01/10.50.0.5:9000.
Already tried 1 time(s).
2014-11-11 04:20:31,872 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: yngcr11hm01/10.50.0.5:9000.
Already tried 2 time(s).
2014-11-11 04:20:32,873 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: yngcr11hm01/10.50.0.5:9000.
Already tried 3 time(s).
2014-11-11 04:20:33,874 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: yngcr11hm01/10.50.0.5:9000.
Already tried 4 time(s).
2014-11-11 04:20:34,875 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: yngcr11hm01/10.50.0.5:9000.
Already tried 5 time(s).
2014-11-11 04:20:35,876 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: yngcr11hm01/10.50.0.5:9000.
Already tried 6 time(s).
2014-11-11 04:20:36,877 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: yngcr11hm01/10.50.0.5:9000.
Already tried 7 time(s).
2014-11-11 04:20:37,877 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: yngcr11hm01/10.50.0.5:9000.
Already tried 8 time(s).
2014-11-11 04:20:38,879 INFO org.apache.hadoop.ipc.Client: Retrying connect to server: yngcr11hm01/10.50.0.5:9000.
Already tried 9 time(s).
2014-11-11 04:20:38,884 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException:
Call to yngcr11hm01/10.50.0.5:9000 failed on local exception: java.net.NoRouteToHostException:
No route to host
	at org.apache.hadoop.ipc.Client.wrapException(Client.java:1107)
	at org.apache.hadoop.ipc.Client.call(Client.java:1075)
	at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
	at com.sun.proxy.$Proxy5.getProtocolVersion(Unknown Source)
	at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
	at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:370)
	at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:429)
	at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:331)
	at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:296)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:356)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:299)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1582)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1521)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1539)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1665)
	at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1682)
Caused by: java.net.NoRouteToHostException: No route to host
	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:597)
	at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
	at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
	at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:434)
	at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:560)
	at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:184)
	at org.apache.hadoop.ipc.Client.getConnection(Client.java:1206)
	at org.apache.hadoop.ipc.Client.call(Client.java:1050)
	... 14 more

2014-11-11 04:20:38,887 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:

/************************************************************
SHUTDOWN_MSG: Shutting down DataNode at yngcr10nc01/10.10.0.10
************************************************************/

ifconfig:
brpriv    Link encap:Ethernet  HWaddr EC:F4:BB:C4:86:2A  
          inet addr:10.100.0.10  Bcast:10.100.255.255  Mask:255.255.0.0
          inet6 addr: fe80::eef4:bbff:fec4:862a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1493 errors:0 dropped:0 overruns:0 frame:0
          TX packets:18 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:73898 (72.1 KiB)  TX bytes:1068 (1.0 KiB)

brpub     Link encap:Ethernet  HWaddr EC:F4:BB:C4:86:28  
          inet addr:10.10.0.10  Bcast:10.10.255.255  Mask:255.255.0.0
          inet6 addr: fe80::eef4:bbff:fec4:8628/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:1274476 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1013239 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:1488923684 (1.3 GiB)  TX bytes:1142276754 (1.0 GiB)

brpub:1   Link encap:Ethernet  HWaddr EC:F4:BB:C4:86:28  
          inet addr:10.10.3.10  Bcast:10.10.255.255  Mask:255.255.0.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1

em1       Link encap:Ethernet  HWaddr EC:F4:BB:C4:86:28  
          inet6 addr: fe80::eef4:bbff:fec4:8628/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:6676964 errors:0 dropped:0 overruns:0 frame:0
          TX packets:1337128 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:5882865885 (5.4 GiB)  TX bytes:1163653184 (1.0 GiB)

em2       Link encap:Ethernet  HWaddr EC:F4:BB:C4:86:2A  
          inet6 addr: fe80::eef4:bbff:fec4:862a/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:16608 errors:0 dropped:0 overruns:0 frame:0
          TX packets:22 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:1004640 (981.0 KiB)  TX bytes:1356 (1.3 KiB)

em3       Link encap:Ethernet  HWaddr EC:F4:BB:C4:86:2C  
          inet addr:10.50.0.10  Bcast:10.50.0.255  Mask:255.255.255.0
          inet6 addr: fe80::eef4:bbff:fec4:862c/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:132716 errors:0 dropped:0 overruns:0 frame:0
          TX packets:103082 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:78672375 (75.0 MiB)  TX bytes:14882729 (14.1 MiB)
          Memory:db680000-db700000 

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:16436  Metric:1
          RX packets:974 errors:0 dropped:0 overruns:0 frame:0
          TX packets:974 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:118148 (115.3 KiB)  TX bytes:118148 (115.3 KiB)

p5p1      Link encap:Ethernet  HWaddr 00:0A:F7:59:34:B0  
          inet addr:10.30.0.10  Bcast:10.30.255.255  Mask:255.255.0.0
          inet6 addr: fe80::20a:f7ff:fe59:34b0/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:39190 errors:0 dropped:0 overruns:0 frame:0
          TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:15408562 (14.6 MiB)  TX bytes:956 (956.0 b)
          Interrupt:40 Memory:d1000000-d17fffff 

p5p2      Link encap:Ethernet  HWaddr 00:0A:F7:59:34:B2  
          inet addr:10.30.2.10  Bcast:10.30.255.255  Mask:255.255.0.0
          inet6 addr: fe80::20a:f7ff:fe59:34b2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:192233 errors:0 dropped:0 overruns:0 frame:0
          TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:12305366 (11.7 MiB)  TX bytes:606 (606.0 b)
          Interrupt:44 Memory:d2000000-d27fffff 

virbr0    Link encap:Ethernet  HWaddr 52:54:00:12:84:4C  
          inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)



Mime
View raw message