hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yongjun Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-7036) HDFS-6776 fix requires to upgrade insecure cluster, which means quite some user pain
Date Wed, 24 Sep 2014 22:08:34 GMT

    [ https://issues.apache.org/jira/browse/HDFS-7036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14146957#comment-14146957

Yongjun Zhang commented on HDFS-7036:

HI [~wheat9], Thanks for your comments. 

Would you mind also comment on how we are going to deal with the similar failure of issuing
hadoop fs -ls <insecureCluster>
from secure cluster side? There is the possibility that other applications too. Given it's
a hack, wonder if you would suggest to have the same hack to different places?

Would you please give some *concrete* example about the potential damage it would cause if
we have the hack in webhdfs? I think that would help the discussion most here.

To me, it's an easy and clean solution to have this hack in webhdfs, so all applications are
taken care of by the solution; and it's going to be easy to take out this hack in the future
when it's the time.

On the other hand, adding the hack in distcp and other applications, the solution in each
application is going to be more complicated than the webhdfs one, and it has the potential
to introduce more instability to the software than the potential damage I can see with the
webhdfs solution. So the *concrete* example I hoped you could give can help the discussion

Thanks a lot.

> HDFS-6776 fix requires to upgrade insecure cluster, which means quite some user pain
> ------------------------------------------------------------------------------------
>                 Key: HDFS-7036
>                 URL: https://issues.apache.org/jira/browse/HDFS-7036
>             Project: Hadoop HDFS
>          Issue Type: Bug
>          Components: webhdfs
>    Affects Versions: 2.5.1
>            Reporter: Yongjun Zhang
>            Assignee: Yongjun Zhang
>         Attachments: HDFS-7036.001.patch
> Issuing command
> {code}
>  hadoop fs -lsr webhdfs://<insecureCluster>
> {code}
> at a secure cluster side fails with message "Failed to get the token ...", similar symptom
as reported in HDFS-6776.
> If the fix of HDFS-6776 is applied to only the secure cluster, doing 
> {code}
> distcp webhdfs://<insecureCluster> <secureCluster>
> {code}
> would fail same way.
> Basically running any application in secure cluster to access insecure cluster via webhdfs
would fail the same way, if the HDFS-6776 fix is not applied to the insecure cluster.
> This could be quite some user pain. Filing this jira for a solution to make user's life
> One proposed solution was to add a msg-parsing mechanism in webhdfs, which is a bit hacky.
The other proposed solution is to do the same kind of hack at application side, which means
the same hack need to be applied in each application.
> Thanks [~daryn], [~wheat9], [~jingzhao], [~tucu00] and [~atm] for the discussion in HDFS-6776.

This message was sent by Atlassian JIRA

View raw message