hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Dahiya (JIRA)" <j...@apache.org>
Subject [jira] Updated: (HADOOP-578) Failed tasks should not be put at the end of the job tracker's queue
Date Thu, 26 Oct 2006 13:25:25 GMT
     [ http://issues.apache.org/jira/browse/HADOOP-578?page=all ]

Sanjay Dahiya updated HADOOP-578:

    Attachment: Hadoop-578.patch

This patch 

- removes firstMapToTry and firstReduceToTry and starts from 0 every time a TaskTracker polls
for new task. 
- removes the extra argument firstTaskToTry from findNewTask(). 

> Failed tasks should not be put at the end of the job tracker's queue
> --------------------------------------------------------------------
>                 Key: HADOOP-578
>                 URL: http://issues.apache.org/jira/browse/HADOOP-578
>             Project: Hadoop
>          Issue Type: Improvement
>          Components: mapred
>    Affects Versions: 0.6.2
>            Reporter: Owen O'Malley
>         Assigned To: Sanjay Dahiya
>         Attachments: Hadoop-578.patch
> This functionality was basically a workaround for other problems, which have been fixed.
The impact of putting them at the end of the work queue is that if you have 100,000 maps,
and the Mapper fails deterministically, you'll run 300,000+ attempts before your job is killed.

This message is automatically generated by JIRA.
If you think it was sent incorrectly contact one of the administrators: http://issues.apache.org/jira/secure/Administrators.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira


View raw message