hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4269) DatanodeManager#registerDatanode rejects all datanode registrations from localhost in single-node developer setup
Date Wed, 05 Dec 2012 04:44:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510273#comment-13510273
] 

Chris Nauroth commented on HDFS-4269:
-------------------------------------

We merged this change to branch-trunk-win on Friday, 11/30. Unfortunately, this had an unintended
side effect of breaking on Windows, at least for single-node developer setups, because of
the code change to reject registration of an unresolved data node:

{code}
public void registerDatanode(DatanodeRegistration nodeReg)
      throws DisallowedDatanodeException {
    InetAddress dnAddress = Server.getRemoteIp();
    if (dnAddress != null) {
      // Mostly called inside an RPC, update ip and peer hostname
      String hostname = dnAddress.getHostName();
      String ip = dnAddress.getHostAddress();
      if (hostname.equals(ip)) {
        LOG.warn("Unresolved datanode registration from " + ip);
        throw new DisallowedDatanodeException(nodeReg);
      }
{code}

On Windows, 127.0.0.1 does not resolve to localhost. It reports host name as "127.0.0.1".
Therefore, on Windows, running pseudo-distributed mode or MiniDFSCluster-based tests always
rejects the datanode registrations. (See HADOOP-8414 for more discussion of the particulars
of resolving 127.0.0.1 on Windows.)

Potential fixes I can think of:
# Add special case logic to allow registration if ip.equals("127.0.0.1"). This is the quick
fix I applied to my dev environment to unblock myself last Friday.
# Add a check against NetUtils.getStaticResolution and register it with NetUtils.addStaticResolution("127.0.0.1",
"localhost") somewhere at initialization time.

Below is a short code sample that demonstrates the problem. This is a very rough approximation
of the IPC Server/Connection and DatanodeManager logic. When I run this server on Mac, it
prints "connection from hostName = localhost, hostAddress = 127.0.0.1, canonicalHostName =
localhost" for any client connection. On Windows, it prints "connection from hostName = 127.0.0.1,
hostAddress = 127.0.0.1, canonicalHostName = 127.0.0.1".

{code}
package cnauroth;

import java.io.PrintWriter;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.ServerSocket;
import java.net.Socket;
import java.nio.channels.ServerSocketChannel;

class Main {
  public static void main(String[] args) throws Exception {
    ServerSocket ss = ServerSocketChannel.open().socket();
    ss.bind(new InetSocketAddress("localhost", 1234), 0);
    System.out.println("ss = " + ss);
    for (;;) {
      Socket s = ss.accept();
      InetAddress addr = s.getInetAddress();
      System.out.println("connection from hostName = " + addr.getHostName() + ", hostAddress
= " + addr.getHostAddress() + ", canonicalHostName = " + addr.getCanonicalHostName());
      PrintWriter pw = new PrintWriter(s.getOutputStream());
      pw.println("hello");
      pw.close();
      s.close();
    }
  }
}
{code}

                
> DatanodeManager#registerDatanode rejects all datanode registrations from localhost in
single-node developer setup
> -----------------------------------------------------------------------------------------------------------------
>
>                 Key: HDFS-4269
>                 URL: https://issues.apache.org/jira/browse/HDFS-4269
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: trunk-win
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
>
> HDFS-3990 is a change that optimized some redundant DNS lookups.  As part of that change,
{{DatanodeManager#registerDatanode}} now rejects attempts to register a datanode for which
the name has not been resolved.  Unfortunately, this broke single-node developer setups on
Windows, because Windows does not resolve 127.0.0.1 to "localhost".

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message