hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Koji Noguchi (JIRA)" <j...@apache.org>
Subject [jira] Created: (HADOOP-2141) speculative execution start up condition based on completion time
Date Thu, 01 Nov 2007 20:03:50 GMT
speculative execution start up condition based on completion time
-----------------------------------------------------------------

                 Key: HADOOP-2141
                 URL: https://issues.apache.org/jira/browse/HADOOP-2141
             Project: Hadoop
          Issue Type: Improvement
          Components: mapred
    Affects Versions: 0.14.3
            Reporter: Koji Noguchi
             Fix For: 0.16.0


We had one job with speculative execution hang.

4 reduce tasks were stuck with 95% completion because of a bad disk. 

Devaraj pointed out 
bq . One of the conditions that must be met for launching a speculative instance of a task
is that it must be at least 20% behind the average progress, and this is not true here.


It would be nice if speculative execution also starts up when tasks stop making progress.

Devaraj suggested 
bq. Maybe, we should introduce a condition for average completion time for tasks in the speculative
execution check. 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message