hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fei Hu <hufe...@gmail.com>
Subject Re: Hadoop Installation on Multihomed Networks
Date Fri, 12 Dec 2014 18:11:42 GMT
I solved the problem by changing the hosts file as follows:
10.10.0.10 10.5.0.10 yngcr10nc01

Thanks,
Fei



> On Nov 11, 2014, at 11:58 AM, daemeon reiydelle <daemeonr@gmail.com> wrote:
> 
> You may want to consider configuring host names that embed the subnet in the host name
itself (e.g. foo50, foo40, for foo via each of the 50 and 40 subnets). ssh key file contents
& etc may have to be fiddled with a bit
> 
> 
> .......
> “The race is not to the swift,
> nor the battle to the strong,
> but to those who can see it coming and jump aside.” - Hunter Thompson
> 
> Daemeon C.M. Reiydelle
> USA (+1) 415.501.0198
> London (+44) (0) 20 8144 9872
> 
> On Tue, Nov 11, 2014 at 7:47 AM, Fei Hu <hufei68@gmail.com <mailto:hufei68@gmail.com>>
wrote:
> Hi,
> 
> I am trying to install Hadoop1.0.4 on multihomed networks. I have done it as the link
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsMultihoming.html
<http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-hdfs/HdfsMultihoming.html>.
But it still doesn’t work.
> 
> The datanode could not work. Its log is as the following. In my hdfs-site.xml, I set
the ip:10.50.0.10 for datanode, but in the following report, host = yngcr10nc01/10.10.0.10
<http://10.10.0.10/>.I think it may be because in /etc/hosts file, I add the pair of
ip and hostname:10.10.0.10 YNGCR10NC01 before. 
> 
> The problem now is that I could not add 10.50.0.10 YNGCR10NC01 into hosts file, because
10.10.0.10 YNGCR10NC01 is necessary for another program.
> 
> Is there any way to solve the problem on multihomed networks?
> 
> Thanks in advance,
> Fei Hu
> 
> 
> 
> 
> 
> 2014-11-11 04:20:28,228 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: STARTUP_MSG:

> /************************************************************
> STARTUP_MSG: Starting DataNode
> STARTUP_MSG:   host = yngcr10nc01/10.10.0.10 <http://10.10.0.10/>
> STARTUP_MSG:   args = []
> STARTUP_MSG:   version = 1.0.4
> STARTUP_MSG:   build = https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0
<https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0> -r 1393290; compiled
by 'hortonfo' on Wed Oct  3 05:13:58 UTC 2012
> ************************************************************/
> 2014-11-11 04:20:28,436 INFO org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties
from hadoop-metrics2.properties
> 2014-11-11 04:20:28,446 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean
for source MetricsSystem,sub=Stats registered.
> 2014-11-11 04:20:28,447 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled
snapshot period at 10 second(s).
> 2014-11-11 04:20:28,447 INFO org.apache.hadoop.metrics2.impl.MetricsSystemImpl: DataNode
metrics system started
> 2014-11-11 04:20:28,572 INFO org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean
for source ugi registered.
> 2014-11-11 04:20:28,607 INFO org.apache.hadoop.util.NativeCodeLoader: Loaded the native-hadoop
library
> 2014-11-11 04:20:29,870 INFO org.apache.hadoop.ipc.Client: Retrying connect to server:
yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/>. Already tried 0 time(s).
> 2014-11-11 04:20:30,871 INFO org.apache.hadoop.ipc.Client: Retrying connect to server:
yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/>. Already tried 1 time(s).
> 2014-11-11 04:20:31,872 INFO org.apache.hadoop.ipc.Client: Retrying connect to server:
yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/>. Already tried 2 time(s).
> 2014-11-11 04:20:32,873 INFO org.apache.hadoop.ipc.Client: Retrying connect to server:
yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/>. Already tried 3 time(s).
> 2014-11-11 04:20:33,874 INFO org.apache.hadoop.ipc.Client: Retrying connect to server:
yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/>. Already tried 4 time(s).
> 2014-11-11 04:20:34,875 INFO org.apache.hadoop.ipc.Client: Retrying connect to server:
yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/>. Already tried 5 time(s).
> 2014-11-11 04:20:35,876 INFO org.apache.hadoop.ipc.Client: Retrying connect to server:
yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/>. Already tried 6 time(s).
> 2014-11-11 04:20:36,877 INFO org.apache.hadoop.ipc.Client: Retrying connect to server:
yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/>. Already tried 7 time(s).
> 2014-11-11 04:20:37,877 INFO org.apache.hadoop.ipc.Client: Retrying connect to server:
yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/>. Already tried 8 time(s).
> 2014-11-11 04:20:38,879 INFO org.apache.hadoop.ipc.Client: Retrying connect to server:
yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/>. Already tried 9 time(s).
> 2014-11-11 04:20:38,884 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: java.io.IOException:
Call to yngcr11hm01/10.50.0.5:9000 <http://10.50.0.5:9000/> failed on local exception:
java.net.NoRouteToHostException: No route to host
> 	at org.apache.hadoop.ipc.Client.wrapException(Client.java:1107)
> 	at org.apache.hadoop.ipc.Client.call(Client.java:1075)
> 	at org.apache.hadoop.ipc.RPC$Invoker.invoke(RPC.java:225)
> 	at com.sun.proxy.$Proxy5.getProtocolVersion(Unknown Source)
> 	at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:396)
> 	at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:370)
> 	at org.apache.hadoop.ipc.RPC.getProxy(RPC.java:429)
> 	at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:331)
> 	at org.apache.hadoop.ipc.RPC.waitForProxy(RPC.java:296)
> 	at org.apache.hadoop.hdfs.server.datanode.DataNode.startDataNode(DataNode.java:356)
> 	at org.apache.hadoop.hdfs.server.datanode.DataNode.<init>(DataNode.java:299)
> 	at org.apache.hadoop.hdfs.server.datanode.DataNode.makeInstance(DataNode.java:1582)
> 	at org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1521)
> 	at org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1539)
> 	at org.apache.hadoop.hdfs.server.datanode.DataNode.secureMain(DataNode.java:1665)
> 	at org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1682)
> Caused by: java.net.NoRouteToHostException: No route to host
> 	at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
> 	at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:597)
> 	at org.apache.hadoop.net.SocketIOWithTimeout.connect(SocketIOWithTimeout.java:206)
> 	at org.apache.hadoop.net.NetUtils.connect(NetUtils.java:489)
> 	at org.apache.hadoop.ipc.Client$Connection.setupConnection(Client.java:434)
> 	at org.apache.hadoop.ipc.Client$Connection.setupIOstreams(Client.java:560)
> 	at org.apache.hadoop.ipc.Client$Connection.access$2000(Client.java:184)
> 	at org.apache.hadoop.ipc.Client.getConnection(Client.java:1206)
> 	at org.apache.hadoop.ipc.Client.call(Client.java:1050)
> 	... 14 more
> 
> 2014-11-11 04:20:38,887 INFO org.apache.hadoop.hdfs.server.datanode.DataNode: SHUTDOWN_MSG:

> /************************************************************
> SHUTDOWN_MSG: Shutting down DataNode at yngcr10nc01/10.10.0.10 <http://10.10.0.10/>
> ************************************************************/
> 
> ifconfig:
> brpriv    Link encap:Ethernet  HWaddr EC:F4:BB:C4:86:2A  
>           inet addr:10.100.0.10  Bcast:10.100.255.255  Mask:255.255.0.0
>           inet6 addr: fe80::eef4:bbff:fec4:862a/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:1493 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:18 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:0 
>           RX bytes:73898 (72.1 KiB)  TX bytes:1068 (1.0 KiB)
> 
> brpub     Link encap:Ethernet  HWaddr EC:F4:BB:C4:86:28  
>           inet addr:10.10.0.10  Bcast:10.10.255.255  Mask:255.255.0.0
>           inet6 addr: fe80::eef4:bbff:fec4:8628/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:1274476 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:1013239 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:0 
>           RX bytes:1488923684 (1.3 GiB)  TX bytes:1142276754 (1.0 GiB)
> 
> brpub:1   Link encap:Ethernet  HWaddr EC:F4:BB:C4:86:28  
>           inet addr:10.10.3.10  Bcast:10.10.255.255  Mask:255.255.0.0
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
> 
> em1       Link encap:Ethernet  HWaddr EC:F4:BB:C4:86:28  
>           inet6 addr: fe80::eef4:bbff:fec4:8628/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:6676964 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:1337128 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:1000 
>           RX bytes:5882865885 (5.4 GiB)  TX bytes:1163653184 (1.0 GiB)
> 
> em2       Link encap:Ethernet  HWaddr EC:F4:BB:C4:86:2A  
>           inet6 addr: fe80::eef4:bbff:fec4:862a/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:16608 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:22 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:1000 
>           RX bytes:1004640 (981.0 KiB)  TX bytes:1356 (1.3 KiB)
> 
> em3       Link encap:Ethernet  HWaddr EC:F4:BB:C4:86:2C  
>           inet addr:10.50.0.10  Bcast:10.50.0.255  Mask:255.255.255.0
>           inet6 addr: fe80::eef4:bbff:fec4:862c/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:132716 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:103082 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:1000 
>           RX bytes:78672375 (75.0 MiB)  TX bytes:14882729 (14.1 MiB)
>           Memory:db680000-db700000 
> 
> lo        Link encap:Local Loopback  
>           inet addr:127.0.0.1  Mask:255.0.0.0
>           inet6 addr: ::1/128 Scope:Host
>           UP LOOPBACK RUNNING  MTU:16436  Metric:1
>           RX packets:974 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:974 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:0 
>           RX bytes:118148 (115.3 KiB)  TX bytes:118148 (115.3 KiB)
> 
> p5p1      Link encap:Ethernet  HWaddr 00:0A:F7:59:34:B0  
>           inet addr:10.30.0.10  Bcast:10.30.255.255  Mask:255.255.0.0
>           inet6 addr: fe80::20a:f7ff:fe59:34b0/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:39190 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:14 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:1000 
>           RX bytes:15408562 (14.6 MiB)  TX bytes:956 (956.0 b)
>           Interrupt:40 Memory:d1000000-d17fffff 
> 
> p5p2      Link encap:Ethernet  HWaddr 00:0A:F7:59:34:B2  
>           inet addr:10.30.2.10  Bcast:10.30.255.255  Mask:255.255.0.0
>           inet6 addr: fe80::20a:f7ff:fe59:34b2/64 Scope:Link
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:192233 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:7 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:1000 
>           RX bytes:12305366 (11.7 MiB)  TX bytes:606 (606.0 b)
>           Interrupt:44 Memory:d2000000-d27fffff 
> 
> virbr0    Link encap:Ethernet  HWaddr 52:54:00:12:84:4C  
>           inet addr:192.168.122.1  Bcast:192.168.122.255  Mask:255.255.255.0
>           UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
>           RX packets:0 errors:0 dropped:0 overruns:0 frame:0
>           TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
>           collisions:0 txqueuelen:0 
>           RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)
> 
> 
> 


Mime
View raw message