hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1389) ApplicationClientProtocol and ApplicationHistoryProtocol should expose analogous APIs
Date Fri, 07 Mar 2014 05:12:47 GMT

    [ https://issues.apache.org/jira/browse/YARN-1389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13923574#comment-13923574
] 

Zhijie Shen commented on YARN-1389:
-----------------------------------

I've tested the patch locally. "yarn applicationattempt" seems to be able to get and list
attempts of RM cached application. "yarn container" is going to result in the following crash:

{code}
zjshen-mac-pc:Deployment zshen$ yarn container -status container_1394168341541_0003_01_000001
14/03/06 21:06:51 INFO client.RMProxy: Connecting to ResourceManager at localhost/127.0.0.1:9104
14/03/06 21:06:51 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your
platform... using builtin-java classes where applicable
14/03/06 21:06:51 INFO client.AHSProxy: Connecting to Application History server at /0.0.0.0:10200
Exception in thread "main" java.lang.NullPointerException: java.lang.NullPointerException
	at org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl.getDiagnosticsInfo(RMContainerImpl.java:253)
	at org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl.createContainerReport(RMContainerImpl.java:439)
	at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getContainerReport(ClientRMService.java:413)
	at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getContainerReport(ApplicationClientProtocolPBServiceImpl.java:364)
	at org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:349)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:605)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:932)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2071)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2067)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:394)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1597)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2065)

	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
	at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
	at org.apache.hadoop.yarn.ipc.RPCUtil.instantiateException(RPCUtil.java:53)
	at org.apache.hadoop.yarn.ipc.RPCUtil.unwrapAndThrowException(RPCUtil.java:107)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getContainerReport(ApplicationClientProtocolPBClientImpl.java:375)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:189)
	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:102)
	at com.sun.proxy.$Proxy10.getContainerReport(Unknown Source)
	at org.apache.hadoop.yarn.client.api.impl.YarnClientImpl.getContainerReport(YarnClientImpl.java:519)
	at org.apache.hadoop.yarn.client.cli.ApplicationCLI.printContainerReport(ApplicationCLI.java:292)
	at org.apache.hadoop.yarn.client.cli.ApplicationCLI.run(ApplicationCLI.java:150)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:84)
	at org.apache.hadoop.yarn.client.cli.ApplicationCLI.main(ApplicationCLI.java:76)
Caused by: org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException): java.lang.NullPointerException
	at org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl.getDiagnosticsInfo(RMContainerImpl.java:253)
	at org.apache.hadoop.yarn.server.resourcemanager.rmcontainer.RMContainerImpl.createContainerReport(RMContainerImpl.java:439)
	at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.getContainerReport(ClientRMService.java:413)
	at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.getContainerReport(ApplicationClientProtocolPBServiceImpl.java:364)
	at org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:349)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:605)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:932)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2071)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2067)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:394)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1597)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2065)

	at org.apache.hadoop.ipc.Client.call(Client.java:1409)
	at org.apache.hadoop.ipc.Client.call(Client.java:1362)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:206)
	at com.sun.proxy.$Proxy9.getContainerReport(Unknown Source)
	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.getContainerReport(ApplicationClientProtocolPBClientImpl.java:372)
	... 13 more
{code}

I checked the code. It seems to be caused by
{code}
  public ContainerStatus getFinishedStatus() {
    return finishedStatus;
  }
{code}

> ApplicationClientProtocol and ApplicationHistoryProtocol should expose analogous APIs
> -------------------------------------------------------------------------------------
>
>                 Key: YARN-1389
>                 URL: https://issues.apache.org/jira/browse/YARN-1389
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Mayank Bansal
>            Assignee: Mayank Bansal
>         Attachments: YARN-1389-1.patch, YARN-1389-2.patch, YARN-1389-3.patch, YARN-1389-4.patch,
YARN-1389-5.patch
>
>
> As we plan to have the APIs in ApplicationHistoryProtocol to expose the reports of *finished*
application attempts and containers, we should do the same for ApplicationClientProtocol,
which will return the reports of *running* attempts and containers.
> Later on, we can improve YarnClient to direct the query of running instance to ApplicationClientProtocol,
while that of finished instance to ApplicationHistoryProtocol, making it transparent to the
users.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message