hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rushabh S Shah (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14445) Delegation tokens are not shared between KMS instances
Date Tue, 26 Dec 2017 23:52:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16304081#comment-16304081
] 

Rushabh S Shah commented on HADOOP-14445:
-----------------------------------------

Thanks [~jojochuang], [~yzhangal] and [~xiaochen] for the great discussion and  review comments
on the first patch.
I have attached trunk and branch-2.8 version of the patch.
Except for checkstyle warnings, the patch is ready to review.
I thought the trunk and branch-2.8 patch would be identical but that is not the case.
There is some difference in {{KMSClientProvider}} because of HADOOP-14987.
The change in HADOOP-14987 is only for debugging purpose and there is no feature or major
enhancement.
@Xiao: Since you are one of the contributor in that jira, we can backport that jira all the
way back to branch-2.8 if you don't mind.

I implemented what we discussed in previous comments.
To summarize let me write below.
* While instantiating {{KMSClientProvider}}, the token service will be initialized to be 
the same as {{CommonConfigurationKeysPublic.HADOOP_SECURITY_KEY_PROVIDER_PATH}}.
* {{KMSTokenRenewer}} will try to instantiate {{KeyProvider}} from token service field.
If it fails to instantiate, it will fallback to the configuration.

I think that should take care of backward compatibility and key shell use cases.
Also added a test case to test backward compatibility for token renewal and cancellation.
I have tried to incorporate all the previous review comments from everyone.
Please let me know if I missed one.

> Delegation tokens are not shared between KMS instances
> ------------------------------------------------------
>
>                 Key: HADOOP-14445
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14445
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: kms
>    Affects Versions: 2.8.0, 3.0.0-alpha1
>         Environment: CDH5.7.4, Kerberized, SSL, KMS-HA, at rest encryption
>            Reporter: Wei-Chiu Chuang
>            Assignee: Rushabh S Shah
>         Attachments: HADOOP-14445-branch-2.8.002.patch, HADOOP-14445-branch-2.8.patch,
HADOOP-14445.002.patch
>
>
> As discovered in HADOOP-14441, KMS HA using LoadBalancingKMSClientProvider do not share
delegation tokens. (a client uses KMS address/port as the key for delegation token)
> {code:title=DelegationTokenAuthenticatedURL#openConnection}
> if (!creds.getAllTokens().isEmpty()) {
>         InetSocketAddress serviceAddr = new InetSocketAddress(url.getHost(),
>             url.getPort());
>         Text service = SecurityUtil.buildTokenService(serviceAddr);
>         dToken = creds.getToken(service);
> {code}
> But KMS doc states:
> {quote}
> Delegation Tokens
> Similar to HTTP authentication, KMS uses Hadoop Authentication for delegation tokens
too.
> Under HA, A KMS instance must verify the delegation token given by another KMS instance,
by checking the shared secret used to sign the delegation token. To do this, all KMS instances
must be able to retrieve the shared secret from ZooKeeper.
> {quote}
> We should either update the KMS documentation, or fix this code to share delegation tokens.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message