hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jian He (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1252) Secure RM fails to start up in secure HA setup with Renewal request for unknown token exception
Date Mon, 30 Sep 2013 21:28:25 GMT

    [ https://issues.apache.org/jira/browse/YARN-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13782263#comment-13782263
] 

Jian He commented on YARN-1252:
-------------------------------

It could be the reason that when the application finishes, NN is failing over and becomes
in SAFEMODE, and at that point of time RM is not able to remove the application state (within
which we store the HDFSDelegationToken) from the store, and RM goes ahead and finishes the
app and add the token to the cancel queue, when new NN is up, the token is canceled. Then
RM shutdown. Since the token is removed on HDFS tokenSecretManager already , when RM comes
back, it will reads the application state(which failed to remove) to try to renew a non-existing
token.

> Secure RM fails to start up in secure HA setup with Renewal request for unknown token
exception
> -----------------------------------------------------------------------------------------------
>
>                 Key: YARN-1252
>                 URL: https://issues.apache.org/jira/browse/YARN-1252
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 2.1.1-beta
>            Reporter: Arpit Gupta
>
> {code}
> 2013-09-26 08:15:20,507 INFO  ipc.Server (Server.java:run(861)) - IPC Server Responder:
starting
> 2013-09-26 08:15:20,521 ERROR security.UserGroupInformation (UserGroupInformation.java:doAs(1486))
- PriviledgedActionException as:rm/host@realm (auth:KERBEROS) cause:org.apache.hadoop.security.token.SecretManager$InvalidToken:
Renewal request for unknown token
>         at org.apache.hadoop.security.token.delegation.AbstractDelegationTokenSecretManager.renewToken(AbstractDelegationTokenSecretManager.java:388)
>         at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.renewDelegationToken(FSNamesystem.java:5934)
>         at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.renewDelegationToken(NameNodeRpcServer.java:453)
>         at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.renewDelegationToken(ClientNamenodeProtocolServerSideTranslatorPB.java:851)
>         at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java:59650)
>         at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)
>         at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2048)
>         at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2044)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at javax.security.auth.Subject.doAs(Subject.java:415)
>         at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1483)
>         at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2042
> {code}



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message