hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-693) Sending NMToken to AM on allocate call
Date Fri, 14 Jun 2013 21:49:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-693?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13683832#comment-13683832

Vinod Kumar Vavilapalli commented on YARN-693:

The MR changes can be moved to YARN-694 or its MR companion.

Some comments on the patch:
 - NMToken should be in org.apache.hadoop.yarn.api.records.
 - Similarly NMTokenPBIMpl in org.apache.hadoop.yarn.api.records.impl.pb.
 - The correct place for calling register and unregister with NMTokenSecretManagerInRM is
 - Static factory for NMToken?
 - And then use the above in NMTokenSecretManagerInRM.
 - AMRMClient signature can just have a ConcurrentMap instead of ConcurrentHashMap.
 - TestAMRMClient: Also validate NMTokens <= nodeCount ?

I suppose that the test to verify that the previously given out tokens before roll-over will
work after roll-over is in YARN-694, right?
> Sending NMToken to AM on allocate call
> --------------------------------------
>                 Key: YARN-693
>                 URL: https://issues.apache.org/jira/browse/YARN-693
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Omkar Vinit Joshi
>            Assignee: Omkar Vinit Joshi
>         Attachments: YARN-693-20130610.patch, YARN-693-20130613.patch
> This is part of YARN-613.
> As per the updated design, AM will receive per NM, NMToken in following scenarios
> * AM is receiving first container on underlying NM.
> * AM is receiving container on underlying NM after either NM or RM rebooted.
> ** After RM reboot, as RM doesn't remember (persist) the information about keys issued
per AM per NM, it will reissue tokens in case AM gets new container on underlying NM. However
on NM side NM will still retain older token until it receives new token to support long running
jobs (in work preserving environment).
> ** After NM reboot, RM will delete the token information corresponding to that AM for
all AMs.
> * AM is receiving container on underlying NM after NMToken master key is rolled over
on RM side.
> In all the cases if AM receives new NMToken then it is suppose to store it for future
NM communication until it receives a new one.
> AMRMClient should expose these NMToken to client. 

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message