hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jian He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3021) YARN's delegation-token handling disallows certain trust setups to operate properly over DistCp
Date Wed, 04 Mar 2015 07:55:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-3021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14346567#comment-14346567

Jian He commented on YARN-3021:

bq. have MR client specify the token renewer it needs to use (instead of your step 1), such
as passing -Dmapreduce.job.delegation.tokenrenewer=null
Are you suggesting a client option to override the renewer for all tokens? the renewer is
on per-token basis; we should not override the renewer for the regular token (the token issued
by its own cluster). What I was suggesting is that we should override the renewer to be null
only for tokens retrieved from the cluster which is different from the cluster where the client

bq. #2 would let certain jobs continue to run even if they would have failed token renewal
without this short term solution.
Actually, instead of checking whether the renewer is RM itself, RM should check if the renewer
is null;  If the renewer is null, RM skips the renew;  otherwise RM can continue renewing
the token; This way a wrong token renewer will also fail the application. The only thing is
that a null renewer would cause application to fail earlier but now will pass, which I think
is fine ?

Also, one problem with current per-app-basis API is that, it'll skip all tokens the application
provides, even though some tokens can  be continued renewed.

> YARN's delegation-token handling disallows certain trust setups to operate properly over
> -----------------------------------------------------------------------------------------------
>                 Key: YARN-3021
>                 URL: https://issues.apache.org/jira/browse/YARN-3021
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: security
>    Affects Versions: 2.3.0
>            Reporter: Harsh J
>         Attachments: YARN-3021.001.patch, YARN-3021.002.patch, YARN-3021.003.patch, YARN-3021.patch
> Consider this scenario of 3 realms: A, B and COMMON, where A trusts COMMON, and B trusts
COMMON (one way trusts both), and both A and B run HDFS + YARN clusters.
> Now if one logs in with a COMMON credential, and runs a job on A's YARN that needs to
access B's HDFS (such as a DistCp), the operation fails in the RM, as it attempts a renewDelegationToken(…)
synchronously during application submission (to validate the managed token before it adds
it to a scheduler for automatic renewal). The call obviously fails cause B realm will not
trust A's credentials (here, the RM's principal is the renewer).
> In the 1.x JobTracker the same call is present, but it is done asynchronously and once
the renewal attempt failed we simply ceased to schedule any further attempts of renewals,
rather than fail the job immediately.
> We should change the logic such that we attempt the renewal but go easy on the failure
and skip the scheduling alone, rather than bubble back an error to the client, failing the
app submission. This way the old behaviour is retained.

This message was sent by Atlassian JIRA

View raw message