hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Joseph Evans (JIRA)" <j...@apache.org>
Subject [jira] [Created] (MAPREDUCE-4775) Reducer will "never" commit suicide
Date Tue, 06 Nov 2012 19:12:12 GMT
Robert Joseph Evans created MAPREDUCE-4775:

             Summary: Reducer will "never" commit suicide
                 Key: MAPREDUCE-4775
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4775
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: mrv2
            Reporter: Robert Joseph Evans
            Assignee: Robert Joseph Evans
            Priority: Critical

In 1.0 there are a number of conditions that will cause a reducer to commit suicide and exit.

This includes if it is stalled, if the error percentage of total fetches is too high.  In
the new code it will only commit suicide when the total number of failures for a single task
attempt is >= max(30, totalMaps/10).  In the best case with the quadratic back-off to get
a single map attempt to reach 30 failure it would take 20.5 hours.  And unless there is only
one reducer running the map task would have been restarted before then.

We should go back to include the same reducer suicide checks that are in 1.0

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message