Mailing-List: contact core-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: core-dev@hadoop.apache.org
Message-ID: <827058844.1214227485118.JavaMail.jira@brutus>
Date: Mon, 23 Jun 2008 06:24:45 -0700 (PDT)
From: "Steve Loughran (JIRA)" <jira@apache.org>
To: core-dev@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-3426) Datanode does not start up if the
 local machines DNS isnt working right and
 dfs.datanode.dns.interface==default
In-Reply-To: <79146276.1211362555756.JavaMail.jira@brutus>
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12607222#action_12607222 ] 

Steve Loughran commented on HADOOP-3426:
----------------------------------------

@work, on a well managed machine with a real IP address in the 0x10, subnet (but unreachable from the outside world)
Testcase: testRDNS took 0.296 sec
	Caused an ERROR
DNS name not found [response code 3]
javax.naming.NameNotFoundException: DNS name not found [response code 3]; remaining name '1.1.0.127.in-addr.arpa'
	at com.sun.jndi.dns.DnsClient.checkResponseCode(DnsClient.java:596)
	at com.sun.jndi.dns.DnsClient.isMatchResponse(DnsClient.java:553)
	at com.sun.jndi.dns.DnsClient.doUdpQuery(DnsClient.java:399)
	at com.sun.jndi.dns.DnsClient.query(DnsClient.java:186)
	at com.sun.jndi.dns.Resolver.query(Resolver.java:64)
	at com.sun.jndi.dns.DnsContext.c_getAttributes(DnsContext.java:413)
	at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_getAttributes(ComponentDirContext.java:213)
	at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.getAttributes(PartialCompositeDirContext.java:121)
	at com.sun.jndi.toolkit.url.GenericURLDirContext.getAttributes(GenericURLDirContext.java:85)
	at javax.naming.directory.InitialDirContext.getAttributes(InitialDirContext.java:123)
	at org.apache.hadoop.net.DNS.reverseDns(DNS.java:67)
	at org.apache.hadoop.net.TestDNS.testRDNS(TestDNS.java:91)

This box does have a proper IPAddr, as resolvable from people.apache.org

> nslookup morzine.hpl.hp.com
Server:         140.211.166.130
Address:        140.211.166.130#53

Non-authoritative answer:
Name:   morzine.hpl.hp.com
Address: 16.25.171.118


but not reverse resolvable, their or elsewhere
> nslookup 118.171.25.16.in-addr.arpa
Server:         140.211.166.131
Address:        140.211.166.131#53

Non-authoritative answer:
*** Can't find 118.171.25.16.in-addr.arpa: No answer

so: 
1. why isnt the full IP address being picked up here? A regression?
2. any code that relies on rDNS to work reliably is in trouble.


> Datanode does not start up if the local machines DNS isnt working right and dfs.datanode.dns.interface==default
> ---------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-3426
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3426
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.16.3, 0.18.0
>         Environment: Ubuntu 8.04, at home, no reverse DNS
>            Reporter: Steve Loughran
>            Priority: Minor
>         Attachments: dns-fixes.patch
>
>
> This is the third Java project I've been involved in that doesnt work on my home network, due to implementation issues with  java.net.InetAddress.getLocalHost(), issues that only show up on an unamanged network. Fortunately my home network exists to find these problems early.
> In hadoop, if the local hostname doesnt resolve, the datanode does not start up:
> Caused by: java.net.UnknownHostException: k2: k2
> at java.net.InetAddress.getLocalHost(InetAddress.java:1353)
> at org.apache.hadoop.net.DNS.getDefaultHost(DNS.java:185)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:184)
> at org.apache.hadoop.dfs.DataNode.(DataNode.java:162)
> at org.apache.hadoop.dfs.ExtDataNode.(ExtDataNode.java:55)
> at org.smartfrog.services.hadoop.components.datanode.DatanodeImpl.sfStart(DatanodeImpl.java:60)
> While this is a valid option in a production (non-virtual) cluster, if you are playing with VMWare/Xen private networks or on a home network, you can't rely on DNS. 
> 1. In these situations, its usually better to fall back to using "localhost" or 127.0.0.1 as a hostname if Java can't work it out for itself,
> 2. Its often good to cache this if used in lots of parts of the system, otherwise the 30s timeouts can cause problems of their own.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.