hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@apache.org>
Subject Re: Distributed hadoop setup 0 live datanode problem in cluster
Date Mon, 22 Mar 2010 11:36:16 GMT
William Kang wrote:
> Hi Jeff,
> I think I partly found out the reasons of this problem. The /etc/hosts
> 127.0.0.1 has the master's host name in it. And the namenode took 127.0.0.1
> as the ip address of the namenode. I fixed it and I already found two nodes.
> There is one still missing. I will let you guys know what happened.
> Thanks.
> 

ongoing issue w/ Ubuntu. It's a feature designed to help laptops and the 
like, useless for when you want to bring up servers visible from the 
outside and addressed by hostname, which is what Hadoop and things like 
Java RMI like.

see http://linux.derkeiler.com/Mailing-Lists/Ubuntu/2007-08/msg00681.html
http://ubuntuforums.org/showthread.php?t=432875
https://lists.ubuntu.com/archives/ubuntu-users/2008-December/168883.html


I've debated fixing this at a pre-hadoop level in my code by having 
checks for the hostname and bailing out early if your hostname maps to 
::1 or 127.*.*.1, because neither are things that are useful off the 
specific host:
http://jira.smartfrog.org/jira/browse/SFOS-1184


Something could go into Hadoop directly, but either way you have a 
problem that hostname/DNS problems can be fairly tricky to identify in a 
timely manner, let alone handle.

-steve

Mime
View raw message