hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ravi Gummadi (Commented) (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-4013) Reduce task gets stuck when a M/R job is configured to tolerate failures
Date Mon, 26 Mar 2012 11:18:26 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-4013?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13238277#comment-13238277

Ravi Gummadi commented on MAPREDUCE-4013:

Patch looks fine to me.
One minor comment:
What about the "progress of map tasks" when there are failed-maps ? Is it getting updated
to 100% ? I see copySucceded() is updating the progress of map-tasks. So what happens when
the last few maps fail ?
> Reduce task gets stuck when a M/R job is configured to tolerate failures
> ------------------------------------------------------------------------
>                 Key: MAPREDUCE-4013
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-4013
>             Project: Hadoop Map/Reduce
>          Issue Type: Bug
>          Components: mrv2
>    Affects Versions: 0.23.2
>            Reporter: Amar Kamat
>            Priority: Blocker
>              Labels: shuffle
>             Fix For: 0.24.0
>         Attachments: MAPREDUCE-4013.patch
> When a M/R job is configured to run with some tolerance to task failures (via mapreduce.map.failures.maxpercent),
then the reduce task of that job gets stuck in the shuffle phase. 

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message