hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HDFS-10757) KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used
Date Thu, 18 Aug 2016 05:32:22 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15425923#comment-15425923
] 

Xiao Chen commented on HDFS-10757:
----------------------------------

Thanks for the comments.

I'm open to changes on how the cache is done (hence also handling the proxy cases Jitendra
mentioned), providing that we thoroughly test to make sure there's no leaking. Looks to me,
HDFS-7718 and HADOOP-11368 are separate issues, which is why HDFS-7718 is done even when HADOOP-11368
is in place.

bq. when the currentUGI is a new proxy user with kms-dt, I don't think we should use the stale
actualUGI here.
The intention of HADOOP-13381 is that, when using a delegation token, the underlying UGI is
bypassed and hence does not matter. See code at [client|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticationHandler.java#L326]
and [server|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/delegation/web/DelegationTokenAuthenticationHandler.java#L326]
side for details.

bq. In a recent change of KMSClientProvider by HADOOP-13155, we can see that the KeyProviderCache
is bypassed
This is not from HADOOP-13155. Token renew/cancellation is done by the [token class with service
loader|https://github.com/apache/hadoop/blob/20f0eb871c57cc4c5a6d19aae0e3745b6175509b/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/token/Token.java#L446],
so HADOOP-13155 is simply hooking that up. The KeyProviderCache is indeed not used, since
this should be done by a service (e.g. yarn), instead of by each client.

> KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used
> -----------------------------------------------------------------------------------
>
>                 Key: HDFS-10757
>                 URL: https://issues.apache.org/jira/browse/HDFS-10757
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>            Assignee: Xiaoyu Yao
>            Priority: Critical
>
> ClientContext::get gets the context from CACHE via a config setting based name, then
KeyProviderCache stored in ClientContext gets the key provider cached by URI from the configuration,
too. These would return the same KeyProvider regardless of current UGI.
> KMSClientProvider caches the UGI (actualUgi) in ctor; that means in particular that all
the users of DFS with KMSClientProvider in a process will get the KMS token (along with other
credentials) of the first user, via the above cache.
> Either KMSClientProvider shouldn't store the UGI, or one of the caches should be UGI-aware,
like the FS object cache.
> Side note: the comment in createConnection that purports to handle the different UGI
doesn't seem to cover what it says it covers. In our case, we have two unrelated UGIs with
no auth (createRemoteUser) with bunch of tokens, including a KMS token, added.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message