hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Haibo Chen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-6885) JobHistory event handling does not complete if handling event throws exception on shutdown
Date Thu, 06 Jul 2017 17:04:00 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-6885?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16076903#comment-16076903
] 

Haibo Chen commented on MAPREDUCE-6885:
---------------------------------------

[~vrushalic] This is what I was referring to that may be related to the issue of no jhist
file that you encountered.

> JobHistory event handling does not complete if handling event throws exception on shutdown
> ------------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-6885
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-6885
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Jonathan Hung
>
> If eventHandlingThread handles an event which causes it to throw an exception (e.g. if
it is unable to flush an event to HDFS), the thread dies. The events are enqueued and eventually
handled when JobHistoryEventHandler stops. If handling these events also throws an exception,
the remaining events are lost. This can for example cause moving job history files to mapreduce.jobhistory.done-dir
to not occur.
> There should be some fail-proof logic here to prevent these events from being lost. Should
also be careful that the same exception is not thrown for each event to prevent the logs from
being cluttered with the same stacktrace. Perhaps we can set a configurable number of failed
handleEvent calls before finally giving up a clean shutdown.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: mapreduce-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: mapreduce-issues-help@hadoop.apache.org


Mime
View raw message