hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1321) NMTokenCache is a a singleton, prevents multiple AMs running in a single JVM to work correctly
Date Sat, 19 Oct 2013 22:23:42 GMT

    [ https://issues.apache.org/jira/browse/YARN-1321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13800014#comment-13800014
] 

Alejandro Abdelnur commented on YARN-1321:
------------------------------------------

The patch does not break existing apps, existing apps, if they don't set a NMTokenCache, will
use the singleton instance.

The changes the patch does in the distributed shell AM are not necessary. I've made them just
as an example of how the new API can be used. I can update the patch reverting them completely.

The changes in the MapReduce AM the only effective change they do is to use a private NMTokenCache
instance. I can change that to use the NMTokenCache singleton instead. This would be the old
behavior. The only changes is that instead using {{NMTokenCache.setNMToken()/getNMToken()}}
it will do {{NMTokenCache.getSingleton().setNMToken()/getNMToken()}}.

I will upload a new patch with the above 2 changes.

> NMTokenCache is a a singleton, prevents multiple AMs running in a single JVM to work
correctly
> ----------------------------------------------------------------------------------------------
>
>                 Key: YARN-1321
>                 URL: https://issues.apache.org/jira/browse/YARN-1321
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: client
>    Affects Versions: 2.2.0
>            Reporter: Alejandro Abdelnur
>            Assignee: Alejandro Abdelnur
>            Priority: Blocker
>             Fix For: 2.2.1
>
>         Attachments: YARN-1321.patch
>
>
> NMTokenCache is a singleton. Because of this, if running multiple AMs in a single JVM
NMTokens for the same node from different AMs step on each other and starting containers fail
due to mismatch tokens.
> The error observed in the client side is something like:
> {code}
> ERROR org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:llama
(auth:PROXY) via llama (auth:SIMPLE) cause:org.apache.hadoop.yarn.exceptions.YarnException:
Unauthorized request to start container. 
> NMToken for application attempt : appattempt_1382038445650_0002_000001 was used for starting
container with container token issued for application attempt : appattempt_1382038445650_0001_000001
> {code}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message