hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haohui Mai (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7036) HDFS-6776 fix requires to upgrade insecure cluster, which means quite some user pain
Date Mon, 23 Mar 2015 21:41:53 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14376700#comment-14376700
] 

Haohui Mai commented on HDFS-7036:
----------------------------------

bq. I answered you with the "hadoop fs -lsr" example. Would you please confirm whether it
addressed your question? Do you think we should fix it too? Thanks.

Fundamentally both distcp and ls are at the application layers. If you want to run ls in secure
clusters to be able to list insecure clusters, then yes, it should be fixed. The problem is
that whether it should be the default behavior, which is up to debate. I can see the values
of both sides of the arguments. Practically it has not been a concern as {{ls}} only involves
one cluster, and it is fairly easy to work around by overriding the configuration at the command
line.

> HDFS-6776 fix requires to upgrade insecure cluster, which means quite some user pain
> ------------------------------------------------------------------------------------
>
>                 Key: HDFS-7036
>                 URL: https://issues.apache.org/jira/browse/HDFS-7036
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: webhdfs
>    Affects Versions: 2.5.1
>            Reporter: Yongjun Zhang
>            Assignee: Yongjun Zhang
>         Attachments: HDFS-7036.001.patch
>
>
> Issuing command
> {code}
>  hadoop fs -lsr webhdfs://<insecureCluster>
> {code}
> at a secure cluster side fails with message "Failed to get the token ...", similar symptom
as reported in HDFS-6776.
> If the fix of HDFS-6776 is applied to only the secure cluster, doing 
> {code}
> distcp webhdfs://<insecureCluster> <secureCluster>
> {code}
> would fail same way.
> Basically running any application in secure cluster to access insecure cluster via webhdfs
would fail the same way, if the HDFS-6776 fix is not applied to the insecure cluster.
> This could be quite some user pain. Filing this jira for a solution to make user's life
easier.
> One proposed solution was to add a msg-parsing mechanism in webhdfs, which is a bit hacky.
The other proposed solution is to do the same kind of hack at application side, which means
the same hack need to be applied in each application.
> Thanks [~daryn], [~wheat9], [~jingzhao], [~tucu00] and [~atm] for the discussion in HDFS-6776.
>  



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message