hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Rushabh S Shah (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-14445) Delegation tokens are not shared between KMS instances
Date Wed, 21 Mar 2018 22:38:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16408698#comment-16408698
] 

Rushabh S Shah commented on HADOOP-14445:
-----------------------------------------

One more high level comment.
 Lets say we are running in an environment where the job submitter and task client are running
with the new code but RM is not upgraded yet.
 Job submitter will have {{KMS_DELEGATION_TOKEN}} and {{kms-dt}} tokens in its credentials
bag.
 Lets say the job ran for more than 24 hours so that RM has to renew the token.
 Since the RM is old, it will only renew {{kms-dt}} token and _will fail to renew KMS_DELEGATION_TOKEN_.
 Now when the new task tries to select the token via {{getKMSToken}}, it will always select
the token with kind {{KMS_DELEGATION_TOKEN}}. 
If I understand the DelegationToken correctly, even if it fails to renew token specific to
\{{KMS_DELEGATION_TOKEN}}, that token will be valid since \{{kms-dt}} is a copy of \{{KMS_D_T}}.
Lets just write a test case to verify.

> Delegation tokens are not shared between KMS instances
> ------------------------------------------------------
>
>                 Key: HADOOP-14445
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14445
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: kms
>    Affects Versions: 2.8.0, 3.0.0-alpha1
>         Environment: CDH5.7.4, Kerberized, SSL, KMS-HA, at rest encryption
>            Reporter: Wei-Chiu Chuang
>            Assignee: Xiao Chen
>            Priority: Major
>         Attachments: HADOOP-14445-branch-2.8.002.patch, HADOOP-14445-branch-2.8.patch,
HADOOP-14445.002.patch, HADOOP-14445.003.patch, HADOOP-14445.004.patch, HADOOP-14445.05.patch,
HADOOP-14445.06.patch, HADOOP-14445.07.patch
>
>
> As discovered in HADOOP-14441, KMS HA using LoadBalancingKMSClientProvider do not share
delegation tokens. (a client uses KMS address/port as the key for delegation token)
> {code:title=DelegationTokenAuthenticatedURL#openConnection}
> if (!creds.getAllTokens().isEmpty()) {
>         InetSocketAddress serviceAddr = new InetSocketAddress(url.getHost(),
>             url.getPort());
>         Text service = SecurityUtil.buildTokenService(serviceAddr);
>         dToken = creds.getToken(service);
> {code}
> But KMS doc states:
> {quote}
> Delegation Tokens
> Similar to HTTP authentication, KMS uses Hadoop Authentication for delegation tokens
too.
> Under HA, A KMS instance must verify the delegation token given by another KMS instance,
by checking the shared secret used to sign the delegation token. To do this, all KMS instances
must be able to retrieve the shared secret from ZooKeeper.
> {quote}
> We should either update the KMS documentation, or fix this code to share delegation tokens.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message