hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Christian Kunz (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3462) reduce task failures during shuffling should not count against number of retry attempts
Date Wed, 28 May 2008 22:28:45 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12600633#action_12600633
] 

Christian Kunz commented on HADOOP-3462:
----------------------------------------

We have a long running job with multiple waves of maps. It occasionally occurs that a relatively
high number of TaskTrackers on nodes with failing disks repeatedly fail, and there number
is high enough to let reduce tasks run out of attempts before these TaskTrackers get blacklisted.

Wouldn't it be better to only count reduce task failures against number of attempts when the
reduce application is actually running, and not already during shuffling?

> reduce task failures during shuffling should not count against number of retry attempts
> ---------------------------------------------------------------------------------------
>
>                 Key: HADOOP-3462
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3462
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>    Affects Versions: 0.16.3
>            Reporter: Christian Kunz
>


-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message