hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Nauroth (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4269) DatanodeManager#registerDatanode rejects all datanode registrations from localhost in single-node developer setup
Date Wed, 05 Dec 2012 04:44:58 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13510273#comment-13510273

Chris Nauroth commented on HDFS-4269:

We merged this change to branch-trunk-win on Friday, 11/30. Unfortunately, this had an unintended
side effect of breaking on Windows, at least for single-node developer setups, because of
the code change to reject registration of an unresolved data node:

public void registerDatanode(DatanodeRegistration nodeReg)
      throws DisallowedDatanodeException {
    InetAddress dnAddress = Server.getRemoteIp();
    if (dnAddress != null) {
      // Mostly called inside an RPC, update ip and peer hostname
      String hostname = dnAddress.getHostName();
      String ip = dnAddress.getHostAddress();
      if (hostname.equals(ip)) {
        LOG.warn("Unresolved datanode registration from " + ip);
        throw new DisallowedDatanodeException(nodeReg);

On Windows, does not resolve to localhost. It reports host name as "".
Therefore, on Windows, running pseudo-distributed mode or MiniDFSCluster-based tests always
rejects the datanode registrations. (See HADOOP-8414 for more discussion of the particulars
of resolving on Windows.)

Potential fixes I can think of:
# Add special case logic to allow registration if ip.equals(""). This is the quick
fix I applied to my dev environment to unblock myself last Friday.
# Add a check against NetUtils.getStaticResolution and register it with NetUtils.addStaticResolution("",
"localhost") somewhere at initialization time.

Below is a short code sample that demonstrates the problem. This is a very rough approximation
of the IPC Server/Connection and DatanodeManager logic. When I run this server on Mac, it
prints "connection from hostName = localhost, hostAddress =, canonicalHostName =
localhost" for any client connection. On Windows, it prints "connection from hostName =,
hostAddress =, canonicalHostName =".

package cnauroth;

import java.io.PrintWriter;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.net.ServerSocket;
import java.net.Socket;
import java.nio.channels.ServerSocketChannel;

class Main {
  public static void main(String[] args) throws Exception {
    ServerSocket ss = ServerSocketChannel.open().socket();
    ss.bind(new InetSocketAddress("localhost", 1234), 0);
    System.out.println("ss = " + ss);
    for (;;) {
      Socket s = ss.accept();
      InetAddress addr = s.getInetAddress();
      System.out.println("connection from hostName = " + addr.getHostName() + ", hostAddress
= " + addr.getHostAddress() + ", canonicalHostName = " + addr.getCanonicalHostName());
      PrintWriter pw = new PrintWriter(s.getOutputStream());

> DatanodeManager#registerDatanode rejects all datanode registrations from localhost in
single-node developer setup
> -----------------------------------------------------------------------------------------------------------------
>                 Key: HDFS-4269
>                 URL: https://issues.apache.org/jira/browse/HDFS-4269
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: namenode
>    Affects Versions: trunk-win
>            Reporter: Chris Nauroth
>            Assignee: Chris Nauroth
> HDFS-3990 is a change that optimized some redundant DNS lookups.  As part of that change,
{{DatanodeManager#registerDatanode}} now rejects attempts to register a datanode for which
the name has not been resolved.  Unfortunately, this broke single-node developer setups on
Windows, because Windows does not resolve to "localhost".

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message