hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiao Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (HADOOP-13381) KMS clients running in the same JVM should use updated KMS Delegation Token
Date Wed, 20 Jul 2016 00:42:20 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-13381?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15385130#comment-15385130

Xiao Chen commented on HADOOP-13381:

bq. the race that multiple threads calling the same cached KMSCP
The problem becomes tougher when considering multi-thread.... The cached {{actualUgi}} is
to handle proxy users, per HADOOP-10698 and HADOOP-11176, so we need that as initial UGI.

For the DT case, we want to pass in the latest credentials. However, the DT-fetching always
happens inside {{actualUgi.doAs}}, which is cached and not updated. I can see the race where
more than 1 thread in comment #1 reaching the same KMSCP, and what we do here would be troublesome.

Don't see a decent solution so far, need more thoughts... Feel free to speak up if any suggestions.

> KMS clients running in the same JVM should use updated KMS Delegation Token
> ---------------------------------------------------------------------------
>                 Key: HADOOP-13381
>                 URL: https://issues.apache.org/jira/browse/HADOOP-13381
>             Project: Hadoop Common
>          Issue Type: Bug
>          Components: kms
>    Affects Versions: 2.6.0
>            Reporter: Xiao Chen
>            Assignee: Xiao Chen
>            Priority: Critical
>         Attachments: HADOOP-13381.01.patch
> When {{/tmp}} is setup as an EZ, one may experience YARN log aggregation failure after
the very first KMS token is expired. The MR job itself runs fine though.
> When this happens, YARN NodeManager's log will show {{AuthenticationException}} with
{{token is expired}} / {{token can't be found in cache}}, depending on whether the expired
token is removed by the background or not.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: common-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: common-issues-help@hadoop.apache.org

View raw message