hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haibo Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6892) Issues with the count of failed/killed tasks in the jhist file
Date Mon, 03 Jul 2017 19:18:02 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16072827#comment-16072827
] 

Haibo Chen commented on MAPREDUCE-6892:
---------------------------------------

Thanks [~pbacsko] for the patch! A few comments: 
1) Now that JobUnsuccessfulCompletionEvent.finishedMaps and JobUnsuccessfulCompletionEvent.finishedReduces
are counters for succeeded maps and reduces, can we rename them to succeededMaps and succeededReduces
to avoid ambiguity? 
2) While at it, it'd great if we also add the new fields to the corresponding timelineevents
in JobUnsuccessfulCompletionEvent.toTimelineEvent().  
3) One more minor issue: remove the redundant new line above JobImpl.unsuccessfulFinish()

> Issues with the count of failed/killed tasks in the jhist file
> --------------------------------------------------------------
>
>                 Key: MAPREDUCE-6892
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6892
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: client, jobhistoryserver
>            Reporter: Peter Bacsko
>            Assignee: Peter Bacsko
>         Attachments: MAPREDUCE-6892-001.patch, MAPREDUCE-6892-002.PATCH
>
>
> Recently we encountered some issues with the value of failed tasks. After parsing the
jhist file, {{JobInfo.getFailedMaps()}} returned 0, but actually there were failures. 
> Another minor thing is that you cannot get the number of killed tasks (although this
can be calculated).
> The root cause is that {{JobUnsuccessfulCompletionEvent}} contains only the successful
map/reduce task counts. Number of failed (or killed) tasks are not stored.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message