hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Zhijie Shen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1445) Separate FINISHING and FINISHED state in YarnApplicationState
Date Tue, 04 Mar 2014 03:43:22 GMT

    [ https://issues.apache.org/jira/browse/YARN-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13918984#comment-13918984

Zhijie Shen commented on YARN-1445:

I've thought about postponing the unregistration success notification until the application
is at FINISHED. It seems impossible, because it will result in a dead lock bellow

1. AM container is waiting for finishing unregistration to move on and exit;
2. Unregistration is waiting for RM notifying success;
3. RM is waiting for RMApp moving from FINISHING to FINISHED to return success;
4. RMApp is waiting for RMAppAttempt moving from FINISHING to FINISHED;
5. RMAppAttempt is waiting for AM container being finished.

Then, if we return a prior state to the client given the internal FINISHING, and still return
unregistration success when RMApp reaches FINISHING, client will see, for example, RUNNING,
while the registration is already successful. The inconsistency here may result in some race
condition for the process relying on checking the final state.

For example, MR client will direct user to AM if the application is said not to be in a final
state. Then, it is possible that AM is unregistered, and RM tells the client that the application
is still running. When the client moves on to contact AM, AM has proceeded and exited before
being able to respond the client request.

It seems that we cannot avoid splitting the user-faced state, and FINISHING can map to a period
of an application's life cycle, which is from unregistration to process exit. [~jlowe] and
[~jianhe], how do you think about it?

> Separate FINISHING and FINISHED state in YarnApplicationState
> -------------------------------------------------------------
>                 Key: YARN-1445
>                 URL: https://issues.apache.org/jira/browse/YARN-1445
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Xuan Gong
>            Assignee: Xuan Gong
>         Attachments: YARN-1445.1.patch, YARN-1445.2.patch, YARN-1445.3.patch, YARN-1445.4.patch,
YARN-1445.5.patch, YARN-1445.5.patch, YARN-1445.6.patch
> Today, we will transmit both RMAppState.FINISHING and RMAppState.FINISHED to YarnApplicationState.FINISHED.

This message was sent by Atlassian JIRA

View raw message