hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Omkar Vinit Joshi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-613) Create NM proxy per NM instead of per container
Date Wed, 15 May 2013 00:36:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13657744#comment-13657744

Omkar Vinit Joshi commented on YARN-613:

I am just summarizing the changes which we need to make for AMNMToken per AM per NM 

AMNMToken will remain valid until application is Alive. So Ideally AM will be able to communicated
with NM as long as
* It received AMNMToken and at least started one container on the underlying Node (NameNode).
* Application has not yet finished.( Because after this NM will no longer remember about this
AMNMToken master key...)

List of changes..
* RM side
** RM will now have ...RMAMNMTokenSecretManager which will generate token for every application
per NM. This token creation will happen only once per NM per AM. If AM requests and gets new
container on same NM then the token will not be regenerated. So RM maintains a map of AMNMTokens
sent per AM per NM ... 
** RM will share master key with NM in its heartbeat if updated.

* AM side
** AM will now have to remember AMNMTokens per NM which it will get only once per NM during
allocate call.
** AM will use this token for authentication by updating UGI while communicating with NM

* NM side
** NMAMNMTokenSecretManager will remember current and previous master key received as a part
of heartbeat.
** It will also remember MasterKeyId per AM (appId) (This is to make sure we can support long
running jobs).
** It will authenticate startContainer, getContainerStatus and stopContainer calls using AMNMToken
via already saved master key. For very first startContainer request for the application using
current/previous master key.

> Create NM proxy per NM instead of per container
> -----------------------------------------------
>                 Key: YARN-613
>                 URL: https://issues.apache.org/jira/browse/YARN-613
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Bikas Saha
>            Assignee: Omkar Vinit Joshi
> Currently a new NM proxy has to be created per container since the secure authentication
is using a containertoken from the container.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message