hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Owen O'Malley (JIRA)" <j...@apache.org>
Subject [jira] Created: (MAPREDUCE-2356) A task succeeded even though there were errors on all attempts.
Date Fri, 04 Mar 2011 17:03:37 GMT
A task succeeded even though there were errors on all attempts.
---------------------------------------------------------------

                 Key: MAPREDUCE-2356
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2356
             Project: Hadoop Map/Reduce
          Issue Type: Bug
            Reporter: Owen O'Malley
            Assignee: Luke Lu
             Fix For: 0.20.100


>From Luke Lu:

Here is a summary of why the failed map task was considered "successful" (Thanks to Mahadev,
Arun and Devaraj
for insightful discussions).

1. The map task was hanging BEFORE being initialized (probably in localization, but it doesn't
matter in this case).
Its state is UNASSIGNED.

2. The jt decided to kill it due to timeout and scheduled a cleanup task on the same node.

3. The cleanup task has the same attempt id (by design.) but runs in a different JVM. Its
initial state is
FAILED_UNCLEAN.

4. The JVM of the original attempt is getting killed, while proceeding to setupWorkDir and
throwed an
IllegalStateException while FileSystem.getLocal, which causes taskFinal.taskCleanup being
called in Child, and
triggered the NPE due to the task is not yet initialized (committer is null). Before the NPE,
however it sent a
statusUpdate to TT, and in tip.reportProgress, changed the task state (currently FAILED_UNCLEAN)
to UNASSIGNED.

5. The cleanup attempt succeeded and report done to TT. In tip.reportDone, the isCleanup()
check returned false due to
the UNASSIGNED state and set the task state as SUCCEEDED.


-- 
This message is automatically generated by JIRA.
-
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message