hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arpit Gupta (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1207) AM fails to register if RM restarts within 5s of job submission
Date Tue, 17 Sep 2013 00:20:51 GMT

    [ https://issues.apache.org/jira/browse/YARN-1207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13769000#comment-13769000
] 

Arpit Gupta commented on YARN-1207:
-----------------------------------

Here is the stack trace from the log

{code}
2013-09-16 21:40:36,852 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server:
hostname/ip:8030. Already tried 9 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50,
sleepTime=1 SECONDS)
2013-09-16 21:40:37,853 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server:
hostname/ip:8030. Already tried 10 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50,
sleepTime=1 SECONDS)
2013-09-16 21:40:38,854 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server:
hostname/ip:8030. Already tried 11 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50,
sleepTime=1 SECONDS)
2013-09-16 21:40:39,855 INFO [main] org.apache.hadoop.ipc.Client: Retrying connect to server:
hostname/ip:8030. Already tried 12 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50,
sleepTime=1 SECONDS)
2013-09-16 21:40:39,895 ERROR [main] org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
as:hrt_qa (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
Password not found for ApplicationAttempt appattempt_1379367602708_0001_000001
2013-09-16 21:40:39,896 WARN [main] org.apache.hadoop.ipc.Client: Exception encountered while
connecting to the server : org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
Password not found for ApplicationAttempt appattempt_1379367602708_0001_000001
2013-09-16 21:40:39,896 ERROR [main] org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
as:hrt_qa (auth:SIMPLE) cause:org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
Password not found for ApplicationAttempt appattempt_1379367602708_0001_000001
2013-09-16 21:40:39,899 ERROR [main] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator:
Exception while registering
org.apache.hadoop.security.token.SecretManager$InvalidToken: Password not found for ApplicationAttempt
appattempt_1379367602708_0001_000001
	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
	at org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53)
	at org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:104)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.registerApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:109)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:601)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:186)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at $Proxy30.registerApplicationMaster(Unknown Source)
	at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.register(RMCommunicator.java:150)
	at org.apache.hadoop.mapreduce.v2.app.rm.RMCommunicator.serviceStart(RMCommunicator.java:110)
	at org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator.serviceStart(RMContainerAllocator.java:213)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$ContainerAllocatorRouter.serviceStart(MRAppMaster.java:790)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
	at org.apache.hadoop.service.CompositeService.serviceStart(CompositeService.java:121)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.serviceStart(MRAppMaster.java:1025)
	at org.apache.hadoop.service.AbstractService.start(AbstractService.java:193)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster$1.run(MRAppMaster.java:1401)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:415)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1483)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.initAndStartAppMaster(MRAppMaster.java:1397)
	at org.apache.hadoop.mapreduce.v2.app.MRAppMaster.main(MRAppMaster.java:1330)
Caused by: org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
Password not found for ApplicationAttempt appattempt_1379367602708_0001_000001
	at org.apache.hadoop.ipc.Client.call(Client.java:1347)
	at org.apache.hadoop.ipc.Client.call(Client.java:1300)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
	at $Proxy29.registerApplicationMaster(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.registerApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:106)
	... 22 more
{code}
                
> AM fails to register if RM restarts within 5s of job submission
> ---------------------------------------------------------------
>
>                 Key: YARN-1207
>                 URL: https://issues.apache.org/jira/browse/YARN-1207
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 2.1.0-beta
>            Reporter: Arpit Gupta
>


--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message