hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5641) History for failed Application Masters should be made available to the Job History Server
Date Mon, 09 Jan 2017 16:39:00 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5641?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15812195#comment-15812195

Jason Lowe commented on MAPREDUCE-5641:

An even simpler approach until we're ready to have the JHS perform REST queries to the AHS
is to have the JHS UI link to the AHS UI.  For example, when the YARN AHS is enabled then
we could make the various app attempt numbers in the MapReduce UI clickable links that go
to the specific attempt page on the AHS UI.  Not as nice as placing the diagnostics directly
on the MapReduce UI, but at least the user can navigate to the information in the interim.

> History for failed Application Masters should be made available to the Job History Server
> -----------------------------------------------------------------------------------------
>                 Key: MAPREDUCE-5641
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5641
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: applicationmaster, jobhistoryserver
>    Affects Versions: 2.2.0
>            Reporter: Robert Kanter
>            Assignee: Robert Kanter
>         Attachments: MAPREDUCE-5641.patch, MAPREDUCE-5641.patch
> Currently, the JHS has no information about jobs whose AMs have failed.  This is because
the History is written by the AM to the intermediate folder just before finishing, so when
it fails for any reason, this information isn't copied there.  However, it is not lost as
its in the AM's staging directory.  To make the History available in the JHS, all we need
to do is have another mechanism to move the History from the staging directory to the intermediate
directory.  The AM also writes a "Summary" file before exiting normally, which is also unavailable
when the AM fails.  

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org

View raw message