hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3103) AMRMClientImpl does not update AMRM token properly
Date Tue, 27 Jan 2015 21:36:34 GMT

    [ https://issues.apache.org/jira/browse/YARN-3103?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14294233#comment-14294233

Jason Lowe commented on YARN-3103:

bq. IIUC, the root cause is that ClientRMProxy#setAMRMTokenService updates the service name
of the token, but doesn't update the service key which is empty.

The key in question is not part of the token but rather part of the Credentials object (i.e.:
the key in the map associated with the token value).  Not sure ClientRMProxy can even update
the key/alias associated with the token in the Credentials since UserGroupInformation either
returns copies of the Credentials or an unmodifiable collection of the tokens.

bq. Is it a valid future use-case that one AM talking with two RMs from separate clusters?
If so, one AM may need to hold two AMRM tokens.

Yes, that's the tricky part.  AFAIK there's no way to know what key/alias a given token is
associated with in the credentials, which makes it particularly tricky to make sure we're
updating the right one.  At least using a non-empty string will help us avoid colliding with
other, non-AMRM tokens accidentally.  The only way I know to fix this the "right" way is if
the RM knew, at the time of the AM launch context creation, what service name the AM would
use to locate the RM.  It could then set the token service name properly before putting it
in the credentials (and not require fixing up the service name by the client).   But currently
I do not believe there is any way for the RM to determine that.

> AMRMClientImpl does not update AMRM token properly
> --------------------------------------------------
>                 Key: YARN-3103
>                 URL: https://issues.apache.org/jira/browse/YARN-3103
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 2.6.0
>            Reporter: Jason Lowe
>            Assignee: Jason Lowe
>            Priority: Blocker
> AMRMClientImpl.updateAMRMToken updates the token service _before_ storing it to the credentials,
so the token is mapped using the newly updated service rather than the empty service that
was used when the RM created the original AMRM token.  This leads to two AMRM tokens in the
credentials and can still fail if the AMRMTokenSelector picks the wrong one.
> In addition the AMRMClientImpl grabs the login user rather than the current user when
security is enabled, so it's likely the UGI being updated is not the UGI that will be used
when reconnecting to the RM.
> The end result is that AMs can fail with invalid token errors when trying to reconnect
to an RM after a new AMRM secret has been activated.

This message was sent by Atlassian JIRA

View raw message