hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3426) Datanode does not start up if the local machines DNS isnt working right and dfs.datanode.dns.interface==default
Date Mon, 23 Jun 2008 13:20:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3426?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12607220#action_12607220
] 

Steve Loughran commented on HADOOP-3426:
----------------------------------------

The reverse DNS test is a test whose failures change with the network infrastructure. This
could be trouble, as anywhere that expects the DNS.reverseDns() method to work reliably will
be in for a disappointment. 

On hudson it is failing with 

javax.naming.NameNotFoundException: DNS name not found [response code 3]; remaining name '106.11.211.140.in-addr.arpa'
	at com.sun.jndi.dns.DnsClient.checkResponseCode(DnsClient.java:596)
	at com.sun.jndi.dns.DnsClient.isMatchResponse(DnsClient.java:553)
	at com.sun.jndi.dns.DnsClient.doUdpQuery(DnsClient.java:399)
	at com.sun.jndi.dns.DnsClient.query(DnsClient.java:186)
	at com.sun.jndi.dns.Resolver.query(Resolver.java:64)
	at com.sun.jndi.dns.DnsContext.c_getAttributes(DnsContext.java:413)
	at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_getAttributes(ComponentDirContext.java:213)
	at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.getAttributes(PartialCompositeDirContext.java:121)
	at com.sun.jndi.toolkit.url.GenericURLDirContext.getAttributes(GenericURLDirContext.java:85)
	at javax.naming.directory.InitialDirContext.getAttributes(InitialDirContext.java:123)
	at org.apache.hadoop.net.DNS.reverseDns(DNS.java:67)
	at org.apache.hadoop.net.TestDNS.testRDNS(TestDNS.java:91)

Standard Output

Localhost IPAddr is localhost/127.0.0.1

and on the apache servers, 
> nslookup 106.11.211.140.in-addr.arpa
Server:         140.211.166.130
Address:        140.211.166.130#53

** server can't find 106.11.211.140.in-addr.arpa: NXDOMAIN



> Datanode does not start up if the local machines DNS isnt working right and dfs.datanode.dns.interface==default
> ---------------------------------------------------------------------------------------------------------------
>
>                 Key: HADOOP-3426
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3426
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: dfs
>    Affects Versions: 0.16.3, 0.18.0
>         Environment: Ubuntu 8.04, at home, no reverse DNS
>            Reporter: Steve Loughran
>            Priority: Minor
>         Attachments: dns-fixes.patch
>
>
> This is the third Java project I've been involved in that doesnt work on my home network,
due to implementation issues with  java.net.InetAddress.getLocalHost(), issues that only show
up on an unamanged network. Fortunately my home network exists to find these problems early.
> In hadoop, if the local hostname doesnt resolve, the datanode does not start up:
> Caused by: java.net.UnknownHostException: k2: k2
> at java.net.InetAddress.getLocalHost(InetAddress.java:1353)
> at org.apache.hadoop.net.DNS.getDefaultHost(DNS.java:185)
> at org.apache.hadoop.dfs.DataNode.startDataNode(DataNode.java:184)
> at org.apache.hadoop.dfs.DataNode.(DataNode.java:162)
> at org.apache.hadoop.dfs.ExtDataNode.(ExtDataNode.java:55)
> at org.smartfrog.services.hadoop.components.datanode.DatanodeImpl.sfStart(DatanodeImpl.java:60)
> While this is a valid option in a production (non-virtual) cluster, if you are playing
with VMWare/Xen private networks or on a home network, you can't rely on DNS. 
> 1. In these situations, its usually better to fall back to using "localhost" or 127.0.0.1
as a hostname if Java can't work it out for itself,
> 2. Its often good to cache this if used in lots of parts of the system, otherwise the
30s timeouts can cause problems of their own.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message