hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xuan Gong (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1410) Handle client failover during 2 step client API's like app submission
Date Sat, 11 Jan 2014 01:03:51 GMT

    [ https://issues.apache.org/jira/browse/YARN-1410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13868557#comment-13868557
] 

Xuan Gong commented on YARN-1410:
---------------------------------

Upload a new patch.
1. move(copy) applicationId duplication check from ClientRMService#submitApplication into
YarnClient#submitApplication (before calling the ClientRMService#submitApplication). So, before
we do submission, we will check duplication first by calling getApplicationReport. If we get
the ApplicationNotFoundException, we will go ahead and submit the application,
2. At ClientRMService#submitApplication, we will have logic to check the duplication. But
this time if we find duplication, that means the corner case that the failover happens before
yarnClient gets SubmitApplicationResponse, but application has been accepted. We will ignore
it and return the SubmitApplicationResponse.

> Handle client failover during 2 step client API's like app submission
> ---------------------------------------------------------------------
>
>                 Key: YARN-1410
>                 URL: https://issues.apache.org/jira/browse/YARN-1410
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Bikas Saha
>            Assignee: Xuan Gong
>         Attachments: YARN-1410-outline.patch, YARN-1410.1.patch, YARN-1410.2.patch
>
>   Original Estimate: 48h
>  Remaining Estimate: 48h
>
> App submission involves
> 1) creating appId
> 2) using that appId to submit an ApplicationSubmissionContext to the user.
> The client may have obtained an appId from an RM, the RM may have failed over, and the
client may submit the app to the new RM.
> Since the new RM has a different notion of cluster timestamp (used to create app id)
the new RM may reject the app submission resulting in unexpected failure on the client side.
> The same may happen for other 2 step client API operations.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message