hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jing Zhao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-5339) WebHDFS URI does not accept logical nameservices when security is enabled
Date Tue, 25 Feb 2014 21:21:28 GMT

    [ https://issues.apache.org/jira/browse/HDFS-5339?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13912080#comment-13912080
] 

Jing Zhao commented on HDFS-5339:
---------------------------------

The patch looks good to me. Some comments:
# It will be better to have a unit test to cover the failure case with HA setup
# We no longer need DTSelecorByKind class after removing its selectToken method
# After removing the selectToken method in DTSelecorByKind, the DT selection will no longer
be able to fall back to use DelegationTokenSelector. Could you please comment why this is
fine and post your system test results?

> WebHDFS URI does not accept logical nameservices when security is enabled
> -------------------------------------------------------------------------
>
>                 Key: HDFS-5339
>                 URL: https://issues.apache.org/jira/browse/HDFS-5339
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: webhdfs
>    Affects Versions: 2.2.0
>            Reporter: Stephen Chu
>            Assignee: Haohui Mai
>         Attachments: HDFS-5339.000.patch, HDFS-5339.001.patch
>
>
> On an insecure, HA setup, we see that this works:
> {code}
> [jenkins@hdfs-cdh5-ha-1 ~]$ hdfs dfs -ls webhdfs://ns1/
> 13/09/27 15:23:52 INFO web.WebHdfsFileSystem: Retrying connect to namenode: hdfs-cdh5-ha-1.ent.cloudera.com/10.20.190.104:20101.
Already tried 0 time(s); retry policy is org.apache.hadoop.io.retry.RetryPolicies$FailoverOnNetworkExceptionRetry@5ebc404e,
delay 0ms.
> Found 5 items
> drwxr-xr-x   - hbase hbase               0 2013-09-23 09:04 webhdfs://ns1/hbase
> drwxrwxr-x   - solr  solr                0 2013-09-18 12:07 webhdfs://ns1/solr
> drwxr-xr-x   - hdfs  supergroup          0 2013-09-19 11:09 webhdfs://ns1/system
> drwxrwxrwt   - hdfs  supergroup          0 2013-09-18 16:25 webhdfs://ns1/tmp
> drwxr-xr-x   - hdfs  supergroup          0 2013-09-18 15:53 webhdfs://ns1/user
> [jenkins@hdfs-cdh5-ha-1 ~]$
> {code}
> However, when security is enabled, we get the following error:
> {code}
> [jenkins@hdfs-cdh5-ha-secure-1 ~]$ hdfs dfs -ls webhdfs://ns1/
> -ls: java.net.UnknownHostException: ns1
> Usage: hadoop fs [generic options] -ls [-d] [-h] [-R] [<path> ...]
> [jenkins@hdfs-cdh5-ha-secure-1 ~]$
> {code}
> I verified that we can use the hdfs://ns1/ URI on the cluster where I see the problem.
> Also, I verified on a secure, non-HA cluster that we can use the webhdfs uri in secure
mode:
> {code}
> [jenkins@hdfs-cdh5-secure-1 ~]$ hdfs dfs -ls webhdfs://hdfs-cdh5-secure-1.ent.cloudera.com:20101/
> drwxr-xr-x   - hbase hbase               0 2013-09-25 10:33 webhdfs://hdfs-cdh5-secure-1.ent.cloudera.com:20101/hbase
> drwxrwxr-x   - solr  solr                0 2013-09-25 10:34 webhdfs://hdfs-cdh5-secure-1.ent.cloudera.com:20101/solr
> drwxrwxrwt   - hdfs  supergroup          0 2013-09-25 10:39 webhdfs://hdfs-cdh5-secure-1.ent.cloudera.com:20101/tmp
> drwxr-xr-x   - hdfs  supergroup          0 2013-09-25 11:00 webhdfs://hdfs-cdh5-secure-1.ent.cloudera.com:20101/user
> [jenkins@hdfs-cdh5-secure-1 ~]$
> {code}



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message