hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Yongjun Zhang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-4564) Webhdfs returns incorrect http response codes for denied operations
Date Tue, 02 Jun 2015 00:55:18 GMT

    [ https://issues.apache.org/jira/browse/HDFS-4564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14568320#comment-14568320
] 

Yongjun Zhang commented on HDFS-4564:
-------------------------------------

HI [~daryn],

Thanks for your earlier work on this jira. I need some help related to this jira, thanks in
advance for your time!

I'm looking into an issue that the following AuthenticationException is thrown when accessing
httpfs via webhdfs in a secure cluster, 
{code}
systest@yj52s ~]$ java -cp $(hadoop classpath):. Foo
15/05/30 00:05:03 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your
platform... using builtin-java classes where applicable
Exception in thread "main" java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException:
Unauthorized
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:309)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$700(WebHdfsFileSystem.java:108)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.getResponse(WebHdfsFileSystem.java:625)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:528)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$200(WebHdfsFileSystem.java:443)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:473)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:470)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.run(WebHdfsFileSystem.java:469)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.run(WebHdfsFileSystem.java:436)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getDelegationToken(WebHdfsFileSystem.java:999)
	at org.apache.hadoop.hdfs.web.TokenAspect.ensureTokenInitialized(TokenAspect.java:140)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getDelegationToken(WebHdfsFileSystem.java:235)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.getAuthParameters(WebHdfsFileSystem.java:393)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.toUrl(WebHdfsFileSystem.java:414)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$FsPathRunner.getUrl(WebHdfsFileSystem.java:649)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.init(WebHdfsFileSystem.java:483)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.runWithRetry(WebHdfsFileSystem.java:524)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.access$200(WebHdfsFileSystem.java:443)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:473)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner$1.run(WebHdfsFileSystem.java:470)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$AbstractRunner.run(WebHdfsFileSystem.java:469)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.run(WebHdfsFileSystem.java:436)
	at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.listStatus(WebHdfsFileSystem.java:982)
	at Foo.main(Foo.java:7)
Caused by: org.apache.hadoop.security.authentication.client.AuthenticationException: Unauthorized
	... 30 more
[systest@yj52s ~]$ 
{code}

The source code Foo.java
{code}
  1 import org.apache.hadoop.conf.*;
  2 import org.apache.hadoop.fs.*;
  3 import org.apache.hadoop.security.*;
  4 class Foo {
  5   public static void main(String args[]) throws Exception {
  6     FileSystem fs = FileSystem.get(new java.net.URI("webhdfs://yj54s.vpc.cloudera.com:14000/"),
new Configuration()); //accessing httpfs via webhdfs
  7     System.out.println(fs.listStatus(new Path("/tmp"))[0]);
  8     java.io.OutputStream os = fs.create(new Path("/tmp/foo"));
  9     os.write('a');
 10     os.close();
 11   }
 12 }
{code}

When I ran without HDFS-4564, it appears to run ok. So the behavior of the above program is
changed by HDFS-4564. I guess it worked because "incorrect http response returned" (which
is fixed by HDFS-4564). 

I traced the execution a bit and I see no "fallback to pseudo auth" happens in this working
run.

Would you please shed some light about your understanding here?

Thanks a lot!




> Webhdfs returns incorrect http response codes for denied operations
> -------------------------------------------------------------------
>
>                 Key: HDFS-4564
>                 URL: https://issues.apache.org/jira/browse/HDFS-4564
>             Project: Hadoop HDFS
>          Issue Type: Sub-task
>          Components: webhdfs
>    Affects Versions: 0.23.0, 2.0.0-alpha, 3.0.0
>            Reporter: Daryn Sharp
>            Assignee: Daryn Sharp
>            Priority: Blocker
>             Fix For: 2.4.0
>
>         Attachments: HDFS-4564.branch-23.patch, HDFS-4564.branch-23.patch, HDFS-4564.branch-23.patch,
HDFS-4564.patch, HDFS-4564.patch
>
>
> Webhdfs is returning 401 (Unauthorized) instead of 403 (Forbidden) when it's denying
operations.  Examples including rejecting invalid proxy user attempts and renew/cancel with
an invalid user.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message