Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Mon, 20 Feb 2017 10:49:44 +0000 (UTC)
From: "Naganarasimha G R (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13043873.1487312091000.9590.1487587784173@Atlassian.JIRA>
In-Reply-To: <JIRA.13043873.1487312091000@Atlassian.JIRA>
References: <JIRA.13043873.1487312091000@Atlassian.JIRA> <JIRA.13043873.1487312091759@jira-lw-us.apache.org>
Subject: [jira] [Commented] (YARN-6207) Move application can  fail when
 attempt add event is delayed
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 20 Feb 2017 10:49:49 -0000


    [ https://issues.apache.org/jira/browse/YARN-6207?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15874336#comment-15874336 ] 

Naganarasimha G R commented on YARN-6207:
-----------------------------------------

Thanks [~rohithsharma] & [~sunilg]
bq. And also we discussed that move application should never-fail if application attempt is not added yet. 
Yes. This is what i was trying to signify by saying once the app is accepted by the scheduler then there should not be any reason to reject the move application. And roughly Me and [~bibinchundatt] were discussing the same as it was easily acheiveable by updating {{SchedulerApplication.setQueue}} at the same we need to ensure that the parent queue stats are updated accordingly. When AppAttempt gets added anyway its taking the queue information from {{SchedulerApplication}} so though attempt is not there when attempt gets added it works properly.

bq. May be new flag is required to identify in SchedulerApplicationAttempt that done-app-attempt is called-off or not.
Agree that there is a corner race case where in 2 Attempts can be added to the target queue when the events which are triggered are {{doneApplicationAttempt->moveApplication-> addApplicationAttempt (new attempt)}}. And for this we can have a flag in SchedulerApplication present in CS. 

Hope with above modifications all concerns will be solved, Thoughts?


> Move application can  fail when attempt add event is delayed
> ------------------------------------------------------------
>
>                 Key: YARN-6207
>                 URL: https://issues.apache.org/jira/browse/YARN-6207
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacity scheduler
>            Reporter: Bibin A Chundatt
>            Assignee: Bibin A Chundatt
>         Attachments: YARN-6207.001.patch, YARN-6207.002.patch
>
>
> *Steps to reproduce*
> 1.Submit application  and delay attempt add to Scheduler
> (Simulate using debug at EventDispatcher for SchedulerEventDispatcher)
> 2. Call move application to destination queue.
> {noformat}
> Caused by: org.apache.hadoop.ipc.RemoteException(java.lang.NullPointerException): java.lang.NullPointerException
> 	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.capacity.CapacityScheduler.preValidateMoveApplication(CapacityScheduler.java:2086)
> 	at org.apache.hadoop.yarn.server.resourcemanager.RMAppManager.moveApplicationAcrossQueue(RMAppManager.java:669)
> 	at org.apache.hadoop.yarn.server.resourcemanager.ClientRMService.moveApplicationAcrossQueues(ClientRMService.java:1231)
> 	at org.apache.hadoop.yarn.api.impl.pb.service.ApplicationClientProtocolPBServiceImpl.moveApplicationAcrossQueues(ApplicationClientProtocolPBServiceImpl.java:388)
> 	at org.apache.hadoop.yarn.proto.ApplicationClientProtocol$ApplicationClientProtocolService$2.callBlockingMethod(ApplicationClientProtocol.java:537)
> 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:522)
> 	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:991)
> 	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:867)
> 	at org.apache.hadoop.ipc.Server$RpcCall.run(Server.java:813)
> 	at java.security.AccessController.doPrivileged(Native Method)
> 	at javax.security.auth.Subject.doAs(Subject.java:422)
> 	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1892)
> 	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2659)
> 	at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1483)
> 	at org.apache.hadoop.ipc.Client.call(Client.java:1429)
> 	at org.apache.hadoop.ipc.Client.call(Client.java:1339)
> 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
> 	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:115)
> 	at com.sun.proxy.$Proxy7.moveApplicationAcrossQueues(Unknown Source)
> 	at org.apache.hadoop.yarn.api.impl.pb.client.ApplicationClientProtocolPBClientImpl.moveApplicationAcrossQueues(ApplicationClientProtocolPBClientImpl.java:398)
> 	... 16 more
> {noformat}


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org