hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-3098) Report Application status as well as ApplicationMaster status in GetApplicationReportResponse
Date Wed, 28 Sep 2011 13:05:46 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-3098?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13116417#comment-13116417

Vinod Kumar Vavilapalli commented on MAPREDUCE-3098:

I understand the areas of concern now. To summarize, let's do this:
 - Rename ApplicationState to _YarnApplicationState_ without changing any functionality.
 - Rename ApplicationState.SUCCEEDED to _YarnApplicationState.FINISHED_
 - Make "String finalState" as "Enum _FinalStatus_" which can take values SUCCEEDED, FAILED

Now the various scenarios:
 - If an application crashes because of Yarn issues, for e.g. AM going over limits, _YarnApplicationState_
will be FAILED.
 - If an application fails because of framework issue, for e.g a MR task fails four times
and thus the job fails, _YarnApplicationState_ will be FINISHED reflecting no platform errors,
but AM will send _FinalStatus_ as FAILED
 - If an application finishes properly and the framework is happy too, for e.g. a successful
MR job, _YarnApplicationState_ will be FINISHED and AM will send _FinalStatus_ to be SUCCEEDED.

To summarize, for the client, the following holds:
||_YarnApplicationState_||_FinalStatus_||What Happenend||
|FINISHED|SUCCEEDED|MR job succeeeded|
|FINISHED|FAILED|MR job failed say due to four attempt failures|
|FINISHED|KILLED|MR job killed by user via AM|
|FAILED|Anything|MR job failed because of YARN errors|
|KILLED|Anything|MR job forcefully terminated via RM|

We will need separate columns on the RM UI for _YarnApplicationState_ and _FinalStatus_. Similarly,
we also need separate columns on the command line for "bin/hadoop job -list" and an extra
entry for _YarnApplicationState_ in "bin/hadoop job -status"
> Report Application status as well as ApplicationMaster status in GetApplicationReportResponse

> ----------------------------------------------------------------------------------------------
>                 Key: MAPREDUCE-3098
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-3098
>             Project: Hadoop Map/Reduce
>          Issue Type: Sub-task
>          Components: mrv2
>    Affects Versions: 0.23.0
>            Reporter: Hitesh Shah
>            Assignee: Hitesh Shah
>            Priority: Blocker
>             Fix For: 0.23.0
> Currently, an application report received by the client from the RM/ASM for a given application
returns the status of the application master. It does not return the status of the application
i.e. whether that particular job succeeded or failed. 
> The AM status would be one of FINISHED (SUCCEEDED should be renamed to FINISHED as AM
state does not indicate overall success/failure), FAILED or KILLED. 
> The final state sent by the AM to the RM in the FinishApplicationMasterRequest should
be exposed to the client as ApplicationState. 

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message