hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gera Shegalov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6315) Implement retrieval of logs for crashed MR-AM via jhist in the staging directory
Date Thu, 26 May 2016 18:52:13 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15302678#comment-15302678
] 

Gera Shegalov commented on MAPREDUCE-6315:
------------------------------------------

Thanks for comments, [~mingma] and [~sjlee0]. I don't get to work on Hadoop as much recently,
and I don't have the bandwidth to work through review iterations right now. This was the minimum
viable patch I implemented more than a year ago at a Hackweek at Twitter. Feel free to take
it over.

As documented in the internal JIRA, making it processed by JHS is a little more complex: some
refactoring to move the "file move" logic from JobHistoryEventHandler#processDoneFiles. I
would not do it in this JIRA.

[~mingma] is correct about the globStatus. jhStats != null might does not seem even a necessary
check for a real glob.

JobHistoryParser output changes are not really required, just wanted to make it prettier with
tabs.



> Implement retrieval of logs for crashed MR-AM via jhist in the staging directory
> --------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6315
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6315
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>          Components: client, mr-am
>    Affects Versions: 2.7.0
>            Reporter: Gera Shegalov
>            Assignee: Gera Shegalov
>            Priority: Critical
>              Labels: BB2015-05-TBR
>         Attachments: MAPREDUCE-6315.001.patch, MAPREDUCE-6315.002.patch, MAPREDUCE-6315.003.patch
>
>
> When all AM attempts crash, there is no record of them in JHS. Thus no easy way to get
the logs. This JIRA automates the procedure by utilizing the jhist file in the staging directory.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message