hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (HADOOP-11017) KMS delegation token secret manager should be able to use zookeeper as store
Date Tue, 23 Sep 2014 19:56:35 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-11017?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Arun Suresh updated HADOOP-11017:
    Attachment: HADOOP-11017.12.patch

Updating patch.. 

Summary of change :
* The {{addPersistedDelegationToken()}} and {{addKey()}} methods are actually called as part
of the {{recover()}} method which is invoked after state is recovered from external store
(ZK) and before the DelegationTokenSecretManager is 'activated' (startThreads() is called).

* Considering the fact that in the data is already in the external store, the patch just modifies
the local {{allKeys}} and {{currentTokens}} maps.
* Patch 11 fixed only the {{addKey()}} method.. Patch 12 takes case of {{addPersistedDelegationToken()}}
as well

[~kasha], I understand your concern about the synchronized block, but if ZK is unavailable,
technically in the Active-Active case (which the ZKDTSM is trying to address).. my opinion
is that this should block the DTSM.. since the update has to be persisted, before proceeding,
else verification of a DelegationToken on a peer node might fail. (Also, prior to the patch,
if you look at the {{createPassword()}} method (which is synchronized)... it used to call
the {{storeNewToken()}} in which the RM state store made a call to ZK)

Ran the following tests in hadoop-yarn to make sure things arn't broken :

 T E S T S

 T E S T S
Running org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 7.158 sec - in org.apache.hadoop.yarn.server.resourcemanager.recovery.TestZKRMStateStore
Running org.apache.hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 5.361 sec - in org.apache.hadoop.yarn.server.resourcemanager.security.TestRMDelegationTokens
Running org.apache.hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 24.719 sec - in org.apache.hadoop.yarn.server.resourcemanager.TestKillApplicationWithRMHA
Running org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart
Tests run: 23, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 189.65 sec - in org.apache.hadoop.yarn.server.resourcemanager.TestRMRestart
Running org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.757 sec - in org.apache.hadoop.yarn.server.resourcemanager.TestSubmitApplicationWithRMHA
Running org.apache.hadoop.yarn.server.resourcemanager.TestWorkPreservingRMRestart

Results :

Tests run: 37, Failures: 0, Errors: 0, Skipped: 0

> KMS delegation token secret manager should be able to use zookeeper as store
> ----------------------------------------------------------------------------
>                 Key: HADOOP-11017
>                 URL: https://issues.apache.org/jira/browse/HADOOP-11017
>             Project: Hadoop Common
>          Issue Type: Improvement
>          Components: security
>    Affects Versions: 2.6.0
>            Reporter: Alejandro Abdelnur
>            Assignee: Arun Suresh
>             Fix For: 2.6.0
>         Attachments: HADOOP-11017.1.patch, HADOOP-11017.10.patch, HADOOP-11017.11.patch,
HADOOP-11017.12.patch, HADOOP-11017.2.patch, HADOOP-11017.3.patch, HADOOP-11017.4.patch, HADOOP-11017.5.patch,
HADOOP-11017.6.patch, HADOOP-11017.7.patch, HADOOP-11017.8.patch, HADOOP-11017.9.patch, HADOOP-11017.WIP.patch
> This will allow supporting multiple KMS instances behind a load balancer.

This message was sent by Atlassian JIRA

View raw message