hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wei-Chiu Chuang (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (HADOOP-14445) Delegation tokens are not shared between KMS instances
Date Wed, 25 Jul 2018 20:44:00 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-14445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556250#comment-16556250
] 

Wei-Chiu Chuang edited comment on HADOOP-14445 at 7/25/18 8:43 PM:
-------------------------------------------------------------------

Had another look a the latest patch (rev14), thanks [~xiaochen] for continuing the work.

Mostly LGTM but the patch won't apply cleanly now due to HADOOP-15591:
{code:java}
LOG.debug("New token received: ({})", token);
{code}
Nit:
 1. why is the setKind() repeated for token?
{code:java|title=TestKMSClientProvider:setup}
	    token.setKind(TOKEN_KIND);
	    token.setService(new Text(providerUriString));
	    token.setKind(TOKEN_KIND);
{code}
Do you meant to set oldToken instead?

2. Question: why is it called “KMSUtilFaultInjector”? It doesn’t emit an exception or
introduce a fault.
{code:java|title=TestLoadBalancingKMSClientProvider#setup()}
KMSUtilFaultInjector.set(new KMSUtilFaultInjector());
{code}
3. it doesn’t look like this line is needed.

4. Finally, how about marking this jira an Incompatible Change? Clearly, this is only an incompatible
change when you flip the switch, however I want to raise a little more awareness of the potential
problem once this switched on.


was (Author: jojochuang):
Had another look a the latest patch (rev14), thanks [~xiaochen] for continuing the work.

Most LGTM but the patch won't apply due to HADOOP-15591:
{code}
LOG.debug("New token received: ({})", token);
{code}

Nit:
1. why is the setKind() repeated for token?
{code:title=TestKMSClientProvider:setup}
	    token.setKind(TOKEN_KIND);
	    token.setService(new Text(providerUriString));
	    token.setKind(TOKEN_KIND);
{code}
Do you meant to set oldToken instead?

2. Question: why is it called “KMSUtilFaultInjector”? It doesn’t emit an exception or
introduce a fault.




{code:title=TestLoadBalancingKMSClientProvider#setup()}
KMSUtilFaultInjector.set(new KMSUtilFaultInjector());
{code}
3. it doesn’t look like this line is needed.

4. Finally, how about marking this jira an Incompatible Change? Clearly, this is only an incompatible
change when you flip the switch, however I want to raise a little more awareness of the potential
problem once this switched on.

> Delegation tokens are not shared between KMS instances
> ------------------------------------------------------
>
>                 Key: HADOOP-14445
>                 URL: https://issues.apache.org/jira/browse/HADOOP-14445
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: kms
>    Affects Versions: 2.8.0, 3.0.0-alpha1
>         Environment: CDH5.7.4, Kerberized, SSL, KMS-HA, at rest encryption
>            Reporter: Wei-Chiu Chuang
>            Assignee: Xiao Chen
>            Priority: Major
>         Attachments: HADOOP-14445-branch-2.8.002.patch, HADOOP-14445-branch-2.8.patch,
HADOOP-14445.002.patch, HADOOP-14445.003.patch, HADOOP-14445.004.patch, HADOOP-14445.05.patch,
HADOOP-14445.06.patch, HADOOP-14445.07.patch, HADOOP-14445.08.patch, HADOOP-14445.09.patch,
HADOOP-14445.10.patch, HADOOP-14445.11.patch, HADOOP-14445.12.patch, HADOOP-14445.13.patch,
HADOOP-14445.14.patch, HADOOP-14445.branch-2.000.precommit.patch, HADOOP-14445.branch-2.001.precommit.patch,
HADOOP-14445.branch-2.01.patch, HADOOP-14445.branch-2.02.patch, HADOOP-14445.branch-2.03.patch,
HADOOP-14445.branch-2.04.patch, HADOOP-14445.branch-2.05.patch, HADOOP-14445.branch-2.06.patch,
HADOOP-14445.branch-2.8.003.patch, HADOOP-14445.branch-2.8.004.patch, HADOOP-14445.branch-2.8.005.patch,
HADOOP-14445.branch-2.8.006.patch, HADOOP-14445.branch-2.8.revert.patch, HADOOP-14445.revert.patch
>
>
> As discovered in HADOOP-14441, KMS HA using LoadBalancingKMSClientProvider do not share
delegation tokens. (a client uses KMS address/port as the key for delegation token)
> {code:title=DelegationTokenAuthenticatedURL#openConnection}
> if (!creds.getAllTokens().isEmpty()) {
>         InetSocketAddress serviceAddr = new InetSocketAddress(url.getHost(),
>             url.getPort());
>         Text service = SecurityUtil.buildTokenService(serviceAddr);
>         dToken = creds.getToken(service);
> {code}
> But KMS doc states:
> {quote}
> Delegation Tokens
> Similar to HTTP authentication, KMS uses Hadoop Authentication for delegation tokens
too.
> Under HA, A KMS instance must verify the delegation token given by another KMS instance,
by checking the shared secret used to sign the delegation token. To do this, all KMS instances
must be able to retrieve the shared secret from ZooKeeper.
> {quote}
> We should either update the KMS documentation, or fix this code to share delegation tokens.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org


Mime
View raw message