hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jian He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4653) Document YARN security model from the perspective of Application Developers
Date Fri, 05 Feb 2016 02:08:40 GMT

    [ https://issues.apache.org/jira/browse/YARN-4653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15133509#comment-15133509

Jian He commented on YARN-4653:

Thanks Steve !Great material !
Some questions/comments I have
bq. It is the responsibility of the application to renew all tokens other than the AMRM and
timeline tokens.
I personally feel here the 'renew' word is a bit confusing. Two kinds of 'renew' we have.
1) Before tokens' final expiration,  tokens submitted via applicaionSubmissionContext are
automatically renewed by DelegationTokenRenwer in RM. 2) After the token final expiration,
application has to re-renew(or 're-fetch') the token by themselves.
Should we clarify these two?
bq. The AM must implement an IPC interface which permits containers to request a new set of
delegation tokens; this interface must itself use authentication and ideally wire encryption.

Wonder how this works.  Since container does not have keytab, so no kerberos channel. What
kind of authentication is this to get the delegation tokens ?
bq. Before a delegation token is due to expire, the processes running in the containers must
request new tokens from the Application Master, over the IPC channel.
Not clear to me how this works. Say, if container wants to get a new hdfs delegation token,
how does it get the new hdfs delegation token from AM? Is it because AM gets a new hdfs delegation
token in the first place which then passed to container when container asks for it? 
bq. Because the RM refreshes tokens on AM restart, 
Correct me if I'm wrong, RM will not refresh any delegation tokens on AM restart. It'll refresh
AMRM token for sure.   
bq.  A thread or executor is started to renew threads on a regular basis.
should it be "is started to renew 'tokens' " ?

> Document YARN security model from the perspective of Application Developers
> ---------------------------------------------------------------------------
>                 Key: YARN-4653
>                 URL: https://issues.apache.org/jira/browse/YARN-4653
>             Project: Hadoop YARN
>          Issue Type: Task
>          Components: site
>    Affects Versions: 2.7.2
>            Reporter: Steve Loughran
>            Assignee: Steve Loughran
>         Attachments: YARN-4653-001.patch, YARN-4653-002.patch
>   Original Estimate: 2h
>  Remaining Estimate: 2h
> What YARN apps need to do for security today is generally copied direct from distributed
shell, with a bit of [ill-informed superstition|https://steveloughran.gitbooks.io/kerberos_and_hadoop/content/sections/yarn.html]
being the sole prose.
> We need a normative document in the YARN site covering
> # the needs for YARN security
> # token creation for AM launch
> # how the RM gets involved
> # token propagation on container launch
> # token renewal strategies
> # How to get tokens for other apps like HBase and Hive.
> # how to work under OOzie
> Perhaps the WritingYarnApplications.md doc is updated, otherwise why not just link to
the relevant bit of the distributed shell client on github for a guarantee of staying up to

This message was sent by Atlassian JIRA

View raw message