hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-613) Create NM proxy per NM instead of per container
Date Wed, 15 May 2013 21:01:18 GMT

    [ https://issues.apache.org/jira/browse/YARN-613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13658816#comment-13658816

Bikas Saha commented on YARN-613:

To be clear, on the AM the behavior is always to take the tokens coming in the allocate response
and setting them in the UGI (overriding old values). They will be picked from the UGI by NMClient
during communication.
The behavior on the NM will be to always authenticate based on the current master key. This
is always the latest correct value and in the majority of the use cases, this master key will
be identical to the cached appId-MasterKey. If the master key matches the incoming token then
the master key is used as the new value of the cached appId-master-key. If the master key
fails to validate the token (long running apps), then the appId-master-key is used to validate
the token.

It would be great to take the solution and break the work into separate jiras. e.g. AMRMProtocol
addition, NMRM protocol changes, RM server changes, NM server changes, AMRMClient changes,
nmclient changes.

bq. If we don't need to preserve the work, (AM and container will be killed after RM restarts)
then there will be no problem at all even with above implementation in which case as applications
are already killed so we can just clear the cache on NM.
If this cache is per appId then it cannot be removed when the appAttempt is completes. It
will be removed when the application completes. During NM resync we should not invalidate
the cache. The cache is required for work preserving restart and will automatically be refreshed
by the above logic for non-work-preserving restart.
> Create NM proxy per NM instead of per container
> -----------------------------------------------
>                 Key: YARN-613
>                 URL: https://issues.apache.org/jira/browse/YARN-613
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Bikas Saha
>            Assignee: Omkar Vinit Joshi
> Currently a new NM proxy has to be created per container since the secure authentication
is using a containertoken from the container.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message