hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hong Tang (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-995) JobHistory should handle cases where task completion events are generated after job completion event
Date Thu, 17 Sep 2009 10:37:57 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-995?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12756466#action_12756466

Hong Tang commented on MAPREDUCE-995:

I found this issue while preparing patch for Mumak (Hadoop Simulator MAPREDUCE-728). The backtrace
of the problem is as follows:
   at org.codehaus.jackson.impl.WriterBasedGenerator._verifyValueWrite(WriterBasedGenerator.java:553)
   at org.codehaus.jackson.impl.JsonGeneratorBase.writeStartObject(JsonGeneratorBase.java:142)
   at org.apache.hadoop.mapreduce.jobhistory.EventWriter.writeEventType(EventWriter.java:81)
   at org.apache.hadoop.mapreduce.jobhistory.EventWriter.write(EventWriter.java:60)
   at org.apache.hadoop.mapreduce.jobhistory.JobHistory.logEvent(JobHistory.java:339)
   at org.apache.hadoop.mapred.JobInProgress.completedTask(JobInProgress.java:2519)
   at org.apache.hadoop.mapred.JobInProgress.updateTaskStatus(JobInProgress.java:1130)
   at org.apache.hadoop.mapred.SimulatorJobTracker.updateTaskStatuses(SimulatorJobTracker.java:621)

The last statement in the backtrace occurs in the context when the job has failed (one task
fails 4 times), and the job is marked as complete. Then a lingering task that belongs to the
same job completes, and JT tries to update its status.

> JobHistory should handle cases where task completion events are generated after job completion
> ----------------------------------------------------------------------------------------------------
>                 Key: MAPREDUCE-995
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-995
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>            Reporter: Jothi Padmanabhan
>            Assignee: Jothi Padmanabhan
> It is apparently possible, in certain circumstances (failed job, for example), for the
job history to get task completion events after the job completion event. This currently causes
NPE in job history.
> Thanks Hong for identifying this issue

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message