hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13988) KMSClientProvider does not work with WebHDFS and Apache Knox w/ProxyUser
Date Thu, 26 Jan 2017 23:09:24 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15840636#comment-15840636
] 

Xiao Chen commented on HADOOP-13988:
------------------------------------

Hm, what I saw on https://builds.apache.org/job/PreCommit-HDFS-Build/18275/testReport/org.apache.hadoop.hdfs/TestAclsEndToEnd/testGoodWithWhitelistWithoutBlacklist/
is sth like this:
{noformat}
2017-01-26 20:32:18,448 ERROR hdfs.TestAclsEndToEnd (TestAclsEndToEnd.java:run(1644)) - IOException
thrown during doAs() operation
java.io.IOException: org.apache.hadoop.security.authentication.client.AuthenticationException:
Authentication failed, URL: http://localhost:36605/kms/v1/keys?doAs=keyadmin&user.name=keyadmin,
status: 403, message: Forbidden
	at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:551)
	at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createKeyInternal(KMSClientProvider.java:732)
	at org.apache.hadoop.crypto.key.kms.KMSClientProvider.createKey(KMSClientProvider.java:742)
	at org.apache.hadoop.crypto.key.KeyProviderExtension.createKey(KeyProviderExtension.java:74)
	at org.apache.hadoop.hdfs.DFSTestUtil.createKey(DFSTestUtil.java:1634)
	at org.apache.hadoop.hdfs.DFSTestUtil.createKey(DFSTestUtil.java:1615)
	at org.apache.hadoop.hdfs.TestAclsEndToEnd$1.execute(TestAclsEndToEnd.java:1532)
	at org.apache.hadoop.hdfs.TestAclsEndToEnd$6.run(TestAclsEndToEnd.java:1640)
	at org.apache.hadoop.hdfs.TestAclsEndToEnd$6.run(TestAclsEndToEnd.java:1636)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:356)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1869)
	at org.apache.hadoop.hdfs.TestAclsEndToEnd.doUserOp(TestAclsEndToEnd.java:1636)
	at org.apache.hadoop.hdfs.TestAclsEndToEnd.createKey(TestAclsEndToEnd.java:1528)
	at org.apache.hadoop.hdfs.TestAclsEndToEnd.doFullAclTest(TestAclsEndToEnd.java:415)
	at org.apache.hadoop.hdfs.TestAclsEndToEnd.testGoodWithWhitelistWithoutBlacklist(TestAclsEndToEnd.java:369)
{noformat}

> KMSClientProvider does not work with WebHDFS and Apache Knox w/ProxyUser
> ------------------------------------------------------------------------
>
>                 Key: HADOOP-13988
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13988
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: common, kms
>    Affects Versions: 2.8.0, 2.7.3
>         Environment: HDP 2.5.3.0 
> WebHDFSUser --> Knox --> HA NameNodes(WebHDFS) --> DataNodes
>            Reporter: Greg Senia
>            Assignee: Xiaoyu Yao
>             Fix For: 2.9.0, 3.0.0-alpha3
>
>         Attachments: HADOOP-13988.01.patch, HADOOP-13988.02.patch, HADOOP-13988.patch,
HADOOP-13988.patch
>
>
> After upgrading to HDP 2.5.3.0 noticed that all of the KMSClientProvider issues have
not been resolved. We put a test build together and applied HADOOP-13558 and HADOOP-13749
these two fixes did still not solve the issue with requests coming from WebHDFS through to
Knox to a TDE zone.
> So we added some debug to our build and determined effectively what is happening here
is a double proxy situation which does not seem to work. So we propose the following fix in
getActualUgi Method:
> {noformat}
>      }
>      // Use current user by default
>      UserGroupInformation actualUgi = currentUgi;
>      if (currentUgi.getRealUser() != null) {
>        // Use real user for proxy user
>        if (LOG.isDebugEnabled()) {
> 	   LOG.debug("using RealUser for proxyUser);
> 	}
>        actualUgi = currentUgi.getRealUser();
>        if (getDoAsUser() != null) {
>       	  if (LOG.isDebugEnabled()) {
> 		LOG.debug("doAsUser exists");
> 		LOG.debug("currentUGI realUser shortName: {}", currentUgi.getRealUser().getShortUserName());
> 		LOG.debug("processUGI loginUser shortName: {}", UserGroupInformation.getLoginUser().getShortUserName());
>           }
>     	  if (currentUgi.getRealUser().getShortUserName() != UserGroupInformation.getLoginUser().getShortUserName())
{
>     		  if (LOG.isDebugEnabled()) {
> 		  	LOG.debug("currentUGI.realUser does not match UGI.processUser);
> 		  }
> 		  actualUgi = UserGroupInformation.getLoginUser();
> 		  if (LOG.isDebugEnabled()) {
> 	    	  	LOG.debug("LoginUser for Proxy: {}", actualUgi.getLoginUser());
> 		  }
>      	  }
>        }
> 	
>      } else if (!currentUgiContainsKmsDt() &&
>          !currentUgi.hasKerberosCredentials()) {
>        // Use login user for user that does not have either
>        // Kerberos credential or KMS delegation token for KMS operations
>        if (LOG.isDebugEnabled()) {
> 	   LOG.debug("using loginUser no KMS Delegation Token no Kerberos Credentials");
> 	}
>        actualUgi = currentUgi.getLoginUser();
>      }
>      return actualUgi;
>    }
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message