hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daryn Sharp (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-60) NMs rejects all container tokens after 2nd secret key roll
Date Wed, 29 Aug 2012 19:59:07 GMT

    [ https://issues.apache.org/jira/browse/YARN-60?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13444364#comment-13444364
] 

Daryn Sharp commented on YARN-60:
---------------------------------

Another datapoint on the first cluster that failed after 2d 15m: left the RM up & bounced
the NMs.  Cluster worked until RM crossed the 3d mark.  Now it's rejecting all container tokens
again.  The 15m activation didn't/shouldn't have occurred so it's more like the second cluster
than failed after 1d.
                
> NMs rejects all container tokens after 2nd secret key roll
> ----------------------------------------------------------
>
>                 Key: YARN-60
>                 URL: https://issues.apache.org/jira/browse/YARN-60
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.2.0-alpha, 0.23.3
>            Reporter: Daryn Sharp
>            Assignee: Vinod Kumar Vavilapalli
>            Priority: Blocker
>
> The NM's token secret manager will reject all container tokens after the third secret
key is activated which means the NM will not launch _any_ containers including AMs.  The whole
yarn cluster becomes inoperable in 2d 15m (default 1d key roll + 15m activation delay).
> The NM tracks the previous 2 secrets, so either the RM is generating tokens with an old
secret, or more likely the NMs are going out of sync when discarding the oldest secret.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message