hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Agarwal (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-13433) webhdfs requests can be routed incorrectly in federated cluster
Date Tue, 17 Apr 2018 18:34:00 GMT

    [ https://issues.apache.org/jira/browse/HDFS-13433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16441313#comment-16441313

Arpit Agarwal commented on HDFS-13433:

bq. Just because the NN has an nsId doesn't mean it overrides the defaultFS authority.
[~daryn], this {{clientNamenodeAddress}} is used for webhdfs redirects. The redirect URL should
use the nameservice that the NN belongs to, and not {{fs.defaultFS}}.

We've seen this in federated clusters, e.g. a webhdfs create request sent to a NameNode in
ns2 uses a redirect url with fs.defaultFS=ns1.

> webhdfs requests can be routed incorrectly in federated cluster
> ---------------------------------------------------------------
>                 Key: HDFS-13433
>                 URL: https://issues.apache.org/jira/browse/HDFS-13433
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Arpit Agarwal
>            Assignee: Arpit Agarwal
>            Priority: Critical
>         Attachments: HDFS-13433.01.patch, HDFS-13433.02.patch, HDFS-13433.03.patch, HDFS-13433.04.patch
> In the following HA+Federated setup with two nameservices ns1 and ns2:
> # ns1 -> namenodes nn1, nn2
> # ns2 -> namenodes nn3, nn4
> # fs.defaultFS is {{hdfs://ns1}}.
> A webhdfs request issued to nn3/nn4 will be routed to ns1. This is because {{setClientNamenodeAddress}}
initializes {{NameNode#clientNamenodeAddress}} using fs.defaultFS before the config is overriden.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org

View raw message