hadoop-hdfs-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiaoyu Yao (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HDFS-10757) KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used
Date Wed, 17 Aug 2016 18:50:20 GMT

    [ https://issues.apache.org/jira/browse/HDFS-10757?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15423519#comment-15423519
] 

Xiaoyu Yao edited comment on HDFS-10757 at 8/17/16 6:49 PM:
------------------------------------------------------------

Thanks [~xiaochen], [~asuresh] and [~jnp] for the discussion. The original issue leaking thread
in the title of HDFS-7718 has been fixed with HADOOP-11368 before HDFS-7718 is resolved. HDFS-7718
introduced KeyProviderCache to the ClientContext. Maybe we should revisit the goal of KeyProviderCache,
which seems to be one of the sources of the problem. KeyProviderCache contains a map with
key based on KMS URI. When combining with KMSClientProvider that caches UGI(actualUgi), wrong
context may be used as the example [~jnp] mentioned above.

HADOOP-13381 changed the KMSClientProvider#createConnection() by checking if the currentUGI
contains kms-dt but only for non-proxy currentUGI. Correct me if I'm wrong: when the currentUGI
is a new proxy user with kms-dt, I don't think we should use the stale actualUGI here. 


was (Author: xyao):
Thanks [~xiaochen], [~asuresh] and [~jnp] for the discussion. The original issue leaking thread
in the title of HDFS-7718 has been fixed with HADOOP-11368 before HDFS-7718 is resolved. HDFS-7718
introduced KeyProviderCache to the ClientContext. Maybe we should revisit the goal of KeyProviderCache,
which seems to be one of the sources of the problem. KeyProviderCache contains a map with
key based on KMS URI. When combining with KMSClientProvider that caches UGI(actualUgi), wrong
context may be used as the example [~jnp] mentioned above.

HADOOP-13381 changed the KMSClientProvider#createConnection() by checking if the currentUGI
contains kms-dt but only for non-proxy currentUGI. Correct me if I'm wrong: when the currentUGI
is a new proxy user with kms-dt, I don't think we should use the stale actualUGI here. Also,
we have a few KMS operations (such as add, and renew/cancel delegation token from HADOOP-13155)
that don't go through KMSClientProvider#createConnection() but use the cached actualUGI. It
will cause similar issue when using with KeyProviderCache enabled. 

> KMSClientProvider combined with KeyProviderCache can result in wrong UGI being used
> -----------------------------------------------------------------------------------
>
>                 Key: HDFS-10757
>                 URL: https://issues.apache.org/jira/browse/HDFS-10757
>             Project: Hadoop HDFS
>          Issue Type: Bug
>            Reporter: Sergey Shelukhin
>            Assignee: Xiaoyu Yao
>            Priority: Critical
>
> ClientContext::get gets the context from CACHE via a config setting based name, then
KeyProviderCache stored in ClientContext gets the key provider cached by URI from the configuration,
too. These would return the same KeyProvider regardless of current UGI.
> KMSClientProvider caches the UGI (actualUgi) in ctor; that means in particular that all
the users of DFS with KMSClientProvider in a process will get the KMS token (along with other
credentials) of the first user, via the above cache.
> Either KMSClientProvider shouldn't store the UGI, or one of the caches should be UGI-aware,
like the FS object cache.
> Side note: the comment in createConnection that purports to handle the different UGI
doesn't seem to cover what it says it covers. In our case, we have two unrelated UGIs with
no auth (createRemoteUser) with bunch of tokens, including a KMS token, added.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: hdfs-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: hdfs-issues-help@hadoop.apache.org


Mime
View raw message