hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1445) Separate FINISHING and FINISHED state in YarnApplicationState
Date Tue, 03 Dec 2013 23:43:35 GMT

    [ https://issues.apache.org/jira/browse/YARN-1445?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13838338#comment-13838338

Jason Lowe commented on YARN-1445:

Personally, I'm still not convinced this is a good idea overall.  If an AM is really doing
things *after* it unregisters in such a way that the client needs to distinguish FINISHING
vs. FINISHED, that itself seems to be a problem.  The AM _must not_ do anything critical after
it unregisters, because there are no retries at that point.  Anything critical needs to be
done _before_ unregistering so if the AM crashes (or its node crashes) those critical steps
will be re-attempted.   In addition there's a limited time in which the app can complete any
of these after-unregister tasks, as the RM will kill it if it takes too long and there's no
way to ask for more time.

There still is the issue of final app time/diagnostics, and in that sense maybe FINISHING
should map to the client state of RUNNING instead of FINISHED.  By exposing this new, unrecognized
state to existing clients, that's essentially how some of them will interpret it if they're
looking for explicit, terminal states (like KILLED/FINISHED/FAILED).  Others will interpret
it as a terminal state since it's not RUNNING.

That being said, if we're still bent on exposing this distinction then here's my take:

- For killing FINISHING apps, I think it's OK to either go ahead and kill the container or
ignore the request.  For the latter case, the RM is going to try to kill it in a few minutes
anyway and forget about it.
- Be sure to check all the existing occurrences of YarnApplication.FINISHED and make sure
we don't need to change that code to handle the new YarnApplication.FINISHING state.

> Separate FINISHING and FINISHED state in YarnApplicationState
> -------------------------------------------------------------
>                 Key: YARN-1445
>                 URL: https://issues.apache.org/jira/browse/YARN-1445
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Xuan Gong
>            Assignee: Xuan Gong
>         Attachments: YARN-1445.1.patch, YARN-1445.2.patch, YARN-1445.3.patch
> Today, we will transmit both RMAppState.FINISHING and RMAppState.FINISHED to YarnApplicationState.FINISHED.

This message was sent by Atlassian JIRA

View raw message