hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Payne (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-1875) MiniDFSCluster hard-codes dfs.datanode.address to localhost
Date Mon, 02 May 2011 21:03:03 GMT
MiniDFSCluster hard-codes dfs.datanode.address to localhost

                 Key: HDFS-1875
                 URL: https://issues.apache.org/jira/browse/HDFS-1875
             Project: Hadoop HDFS
          Issue Type: Bug
          Components: test
    Affects Versions: 0.22.0
            Reporter: Eric Payne
            Assignee: Eric Payne
             Fix For: 0.23.0

When creating RPC addresses that represent the communication sockets for each simulated DataNode,
the MiniDFSCluster class hard-codes the address of the dfs.datanode.address port to be ""

The DataNodeCluster test tool uses the MiniDFSCluster class to create a selected number of
simulated datanodes on a single host. In the DataNodeCluster setup, the NameNode is not simulated
but is started as a separate daemon.

The problem is that if the write requrests into the simulated datanodes are originated on
a host that is not the same host running the simulated datanodes, the connections are refused.
This is because the RPC sockets that are started by MiniDFSCluster are for "localhost" (
and are not accessible from outside that same machine.

It is proposed that the MiniDFSCluster.setupDatanodeAddress() method be overloaded in order
to accommodate an environment where the NameNode is on one host, the client is on another
host, and the simulated DataNodes are on yet another host (or even multiple hosts simulating
multiple DataNodes each).

The overloaded API would add a parameter that would be used as the basis for creating the
RPS sockets. By default, it would remain

This message is automatically generated by JIRA.
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message