hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oleg Zhurakousky <oleg.zhurakou...@gmail.com>
Subject Re: Question about YARN security tokens
Date Fri, 25 Apr 2014 11:53:13 GMT
Without looking at the code its hard to say. Perhaps looking at a working
code will put you in the right direction.
For example, here is the DistributedShell from Hadoop (only few classes)
https://github.com/apache/hadoop-common/tree/trunk/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell
That is what I used a a model to dig into some of the YARN internals as I
am playing around with ideas on how to simplify YARN API
https://github.com/olegz/yaya, so you can use that code as well to see if
you can get some more information from it.

Cheers
Oleg


On Fri, Apr 25, 2014 at 12:22 AM, Robert Chu <allegrormc@gmail.com> wrote:

> Hi Everyone,
>
> I'm new to YARN and was trying to write a simple YARN application that
> starts up an ApplicationMaster that starts up an in-process Jetty server
> just a simple test (this application requests no further resource
> containers). An attempt is made to register as an application with the
> ResourceManager while starting the ApplicationMaster process but this fails
> with an exception related to security tokens when using the MiniYARNCluster:
>
> Exception in thread "main"
> org.apache.hadoop.security.token.SecretManager$InvalidToken: Password not
> found for ApplicationAttempt appattempt_1398290928870_0001_000002
>  at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
>  at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
>  at
> org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53)
> at
> org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:104)
>  at
> org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.registerApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:109)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>  at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>  at java.lang.reflect.Method.invoke(Method.java:606)
> at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:186)
>  at
> org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
> at com.sun.proxy.$Proxy8.registerApplicationMaster(Unknown Source)
>  at
> org.apache.hadoop.yarn.client.api.impl.AMRMClientImpl.registerApplicationMaster(AMRMClientImpl.java:196)
> at
> org.apache.hadoop.yarn.client.api.async.impl.AMRMClientAsyncImpl.registerApplicationMaster(AMRMClientAsyncImpl.java:138)
>  at
> org.kiji.scoring.server.yarn.YarnServiceMaster.start(YarnServiceMaster.java:228)
> at
> org.kiji.scoring.server.yarn.YarnServiceMaster.main(YarnServiceMaster.java:333)
> Caused by:
> org.apache.hadoop.ipc.RemoteException(org.apache.hadoop.security.token.SecretManager$InvalidToken):
> Password not found for ApplicationAttempt
> appattempt_1398290928870_0001_000002
> at org.apache.hadoop.ipc.Client.call(Client.java:1409)
>  at org.apache.hadoop.ipc.Client.call(Client.java:1362)
> at
> org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
>  at com.sun.proxy.$Proxy7.registerApplicationMaster(Unknown Source)
> at
> org.apache.hadoop.yarn.api.impl.pb.client.ApplicationMasterProtocolPBClientImpl.registerApplicationMaster(ApplicationMasterProtocolPBClientImpl.java:106)
>  ... 11 more
>
> at org.apache.hadoop.util.Shell.runCommand(Shell.java:505)
> at org.apache.hadoop.util.Shell.run(Shell.java:418)
>  at
> org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
> at
> org.apache.hadoop.yarn.server.nodemanager.DefaultContainerExecutor.launchContainer(DefaultContainerExecutor.java:195)
>  at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:283)
> at
> org.apache.hadoop.yarn.server.nodemanager.containermanager.launcher.ContainerLaunch.call(ContainerLaunch.java:79)
>  at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
> at java.util.concurrent.FutureTask.run(FutureTask.java:138)
>  at
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
>  at java.lang.Thread.run(Thread.java:662)
>
> Upon getting this stack trace, I attempted to find
> documentation/information/examples about security tokens but was unable to
> find anything of note.
>
> Am I doing something dumb here and is this not related to security tokens?
> If not, is there documentation or good examples around how to use security
> tokens while launching an application?
>
> Thanks!
>
> Robert Chu
>

Mime
View raw message