hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Philip Zeyliger (JIRA)" <j...@apache.org>
Subject [jira] [Created] (HDFS-13797) WebHDFS redirects to datanodes by hostname, ignoring bind addresses
Date Mon, 06 Aug 2018 20:25:00 GMT
Philip Zeyliger created HDFS-13797:

             Summary: WebHDFS redirects to datanodes by hostname, ignoring bind addresses
                 Key: HDFS-13797
                 URL: https://issues.apache.org/jira/browse/HDFS-13797
             Project: Hadoop HDFS
          Issue Type: Task
          Components: webhdfs
            Reporter: Philip Zeyliger

In testing Impala, we run datanodes bound to {{}}. This works great, except for WebHDFS,
where the namenode redirects to a datanode by hostname, leading to a "connection refused"
error (since the datanode isn't listening). Our workaround in Impala is to map hostname to
{{}} in {{/etc/hosts}} which isn't so hot. (The workaround is at https://github.com/apache/impala/blob/3334c167a69a6506d595898dff73fb16410266e6/bin/bootstrap_system.sh#L187

I think there's a bug here. Reading https://hadoop.apache.org/docs/r3.0.3/hadoop-project-dist/hadoop-hdfs/HdfsMultihoming.html,
it looks like WebHDFS should perhaps respect {{dfs.client.use.datanode.hostname}}, but I think
it doesn't and always uses the hostname. I think the relevant point where the namenode figures
this out is https://github.com/apache/hadoop/blob/ca20e0d7e9767a7362dddfea8ec19548947d3fd7/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/web/resources/NamenodeWebHdfsMethods.java#L406

I recognize this is a non-production edge case: it makes no sense to listen only on localhost
except for testing.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-dev-help@hadoop.apache.org

View raw message