hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Greg Senia (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13988) KMSClientProvider does not work with WebHDFS and Apache Knox w/ProxyUser
Date Fri, 20 Jan 2017 01:48:27 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13988?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15831010#comment-15831010
] 

Greg Senia commented on HADOOP-13988:
-------------------------------------

[~lmccay]  the logs from above are from the patch uploaded an hour ago. Let me know if it
looks like code path is wrong from what I can see the code path is working correctly and the
!equals is definitely working correctly if it wasn't it would of failed.


Also here is the patch output from my last build about an hour ago with the updated path from
today:

ETG-GSeni-MBP:hadoop-release gss2002$ patch -p1 < ../../kmsfixes/HADOOP-13558.02.patch

patching file hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
Hunk #1 succeeded at 618 with fuzz 1 (offset -14 lines).
Hunk #2 succeeded at 825 (offset -40 lines).
patching file hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/security/TestUserGroupInformation.java
Hunk #1 succeeded at 31 (offset -1 lines).
Hunk #2 succeeded at 902 with fuzz 2 (offset -111 lines).




ETG-GSeni-MBP:hadoop-release gss2002$ patch -p1 < ../../kmsfixes/HADOOP-13749.00.patch

patching file hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSClientProvider.java
Hunk #4 succeeded at 901 (offset 2 lines).
Hunk #5 succeeded at 924 (offset 2 lines).
Hunk #6 succeeded at 996 (offset 2 lines).
Hunk #7 succeeded at 1042 (offset 2 lines).
patching file hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
Hunk #1 succeeded at 1768 (offset -55 lines).
patching file hadoop-common-project/hadoop-kms/src/test/java/org/apache/hadoop/crypto/key/kms/server/TestKMS.java
Hunk #1 succeeded at 1825 (offset -8 lines).
Hunk #2 succeeded at 2149 (offset -5 lines).


ETG-GSeni-MBP:hadoop-release gss2002$ patch -p1 < ../../HADOOP-13988.patch 
patching file hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/crypto/key/kms/KMSClientProvider.java
Hunk #1 succeeded at 1052 (offset -10 lines).
patching file hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/UserGroupInformation.java
Hunk #1 succeeded at 1774 (offset -67 lines).

> KMSClientProvider does not work with WebHDFS and Apache Knox w/ProxyUser
> ------------------------------------------------------------------------
>
>                 Key: HADOOP-13988
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13988
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: common, kms
>    Affects Versions: 2.8.0, 2.7.3
>         Environment: HDP 2.5.3.0 
> WebHDFSUser --> Knox --> HA NameNodes(WebHDFS) --> DataNodes
>            Reporter: Greg Senia
>         Attachments: HADOOP-13988.patch, HADOOP-13988.patch
>
>
> After upgrading to HDP 2.5.3.0 noticed that all of the KMSClientProvider issues have
not been resolved. We put a test build together and applied HADOOP-13558 and HADOOP-13749
these two fixes did still not solve the issue with requests coming from WebHDFS through to
Knox to a TDE zone.
> So we added some debug to our build and determined effectively what is happening here
is a double proxy situation which does not seem to work. So we propose the following fix in
getActualUgi Method:
> {noformat}
>      }
>      // Use current user by default
>      UserGroupInformation actualUgi = currentUgi;
>      if (currentUgi.getRealUser() != null) {
>        // Use real user for proxy user
>        if (LOG.isDebugEnabled()) {
> 	   LOG.debug("using RealUser for proxyUser);
> 	}
>        actualUgi = currentUgi.getRealUser();
>        if (getDoAsUser() != null) {
>       	  if (LOG.isDebugEnabled()) {
> 		LOG.debug("doAsUser exists");
> 		LOG.debug("currentUGI realUser shortName: {}", currentUgi.getRealUser().getShortUserName());
> 		LOG.debug("processUGI loginUser shortName: {}", UserGroupInformation.getLoginUser().getShortUserName());
>           }
>     	  if (currentUgi.getRealUser().getShortUserName() != UserGroupInformation.getLoginUser().getShortUserName())
{
>     		  if (LOG.isDebugEnabled()) {
> 		  	LOG.debug("currentUGI.realUser does not match UGI.processUser);
> 		  }
> 		  actualUgi = UserGroupInformation.getLoginUser();
> 		  if (LOG.isDebugEnabled()) {
> 	    	  	LOG.debug("LoginUser for Proxy: {}", actualUgi.getLoginUser());
> 		  }
>      	  }
>        }
> 	
>      } else if (!currentUgiContainsKmsDt() &&
>          !currentUgi.hasKerberosCredentials()) {
>        // Use login user for user that does not have either
>        // Kerberos credential or KMS delegation token for KMS operations
>        if (LOG.isDebugEnabled()) {
> 	   LOG.debug("using loginUser no KMS Delegation Token no Kerberos Credentials");
> 	}
>        actualUgi = currentUgi.getLoginUser();
>      }
>      return actualUgi;
>    }
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message