hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiaoyu Yao (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13988) KMSClientProvider does not work with WebHDFS and Apache Knox w/ProxyUser
Date Wed, 25 Jan 2017 03:14:26 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15837106#comment-15837106
] 

Xiaoyu Yao commented on HADOOP-13988:
-------------------------------------

Thanks [~jnp] and [~lmccay] for the review. 

bq. Knox never interacts directly with KMS and neither does the Knox enduser.

Hadoop proxy user does not recommend using delegation token to proxy another user. Oozie for
example uses a kerberos to proxy its end user. That's also the expected usage from HADOOP-13749.


Knox can either uses UGI with kerberos to create proxy user for its end user, or impersonate
end user to get KMS-DT and add it to the end user's UGI if the file accessed is in encryption
zone.

> KMSClientProvider does not work with WebHDFS and Apache Knox w/ProxyUser
> ------------------------------------------------------------------------
>
>                 Key: HADOOP-13988
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13988
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: common, kms
>    Affects Versions: 2.8.0, 2.7.3
>         Environment: HDP 2.5.3.0 
> WebHDFSUser --> Knox --> HA NameNodes(WebHDFS) --> DataNodes
>            Reporter: Greg Senia
>            Assignee: Xiaoyu Yao
>         Attachments: HADOOP-13988.01.patch, HADOOP-13988.02.patch, HADOOP-13988.patch,
HADOOP-13988.patch
>
>
> After upgrading to HDP 2.5.3.0 noticed that all of the KMSClientProvider issues have
not been resolved. We put a test build together and applied HADOOP-13558 and HADOOP-13749
these two fixes did still not solve the issue with requests coming from WebHDFS through to
Knox to a TDE zone.
> So we added some debug to our build and determined effectively what is happening here
is a double proxy situation which does not seem to work. So we propose the following fix in
getActualUgi Method:
> {noformat}
>      }
>      // Use current user by default
>      UserGroupInformation actualUgi = currentUgi;
>      if (currentUgi.getRealUser() != null) {
>        // Use real user for proxy user
>        if (LOG.isDebugEnabled()) {
> 	   LOG.debug("using RealUser for proxyUser);
> 	}
>        actualUgi = currentUgi.getRealUser();
>        if (getDoAsUser() != null) {
>       	  if (LOG.isDebugEnabled()) {
> 		LOG.debug("doAsUser exists");
> 		LOG.debug("currentUGI realUser shortName: {}", currentUgi.getRealUser().getShortUserName());
> 		LOG.debug("processUGI loginUser shortName: {}", UserGroupInformation.getLoginUser().getShortUserName());
>           }
>     	  if (currentUgi.getRealUser().getShortUserName() != UserGroupInformation.getLoginUser().getShortUserName())
{
>     		  if (LOG.isDebugEnabled()) {
> 		  	LOG.debug("currentUGI.realUser does not match UGI.processUser);
> 		  }
> 		  actualUgi = UserGroupInformation.getLoginUser();
> 		  if (LOG.isDebugEnabled()) {
> 	    	  	LOG.debug("LoginUser for Proxy: {}", actualUgi.getLoginUser());
> 		  }
>      	  }
>        }
> 	
>      } else if (!currentUgiContainsKmsDt() &&
>          !currentUgi.hasKerberosCredentials()) {
>        // Use login user for user that does not have either
>        // Kerberos credential or KMS delegation token for KMS operations
>        if (LOG.isDebugEnabled()) {
> 	   LOG.debug("using loginUser no KMS Delegation Token no Kerberos Credentials");
> 	}
>        actualUgi = currentUgi.getLoginUser();
>      }
>      return actualUgi;
>    }
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message