hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Amareshwari Sriramadasu (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-4246) Reduce task copy errors may not kill it eventually
Date Tue, 30 Sep 2008 05:45:46 GMT

     [ https://issues.apache.org/jira/browse/HADOOP-4246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Amareshwari Sriramadasu updated HADOOP-4246:
--------------------------------------------

    Attachment: patch-4246.txt

Here is a patch doing :
1. maxFetchRetriesPerMap is assigned value 1, if it is zero.
2. maxFailedUniqueFetches is assigned value of numMaps, if numMaps is less than 5.

Tested the patch by throwing an FSError from copyOutput, with numMaps = 3 and mapRunTime=2secs.


> Reduce task copy errors may not kill it eventually
> --------------------------------------------------
>
>                 Key: HADOOP-4246
>                 URL: https://issues.apache.org/jira/browse/HADOOP-4246
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.19.0
>            Reporter: Amareshwari Sriramadasu
>            Assignee: Amareshwari Sriramadasu
>            Priority: Blocker
>             Fix For: 0.19.0
>
>         Attachments: patch-4246.txt, patch-4246.txt
>
>
> maxFetchRetriesPerMap in reduce task can be zero some times (when maxMapRunTime is less
than 4 seconds or mapred.reduce.copy.backoff is less than 4). This will not count reduce task
copy errors to kill it eventually.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message