hadoop-mapreduce-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Joydeep Sen Sarma (JIRA)" <j...@apache.org>
Subject [jira] Created: (MAPREDUCE-2175) speculation framework does not update job progress rate with time causing false speculation
Date Wed, 03 Nov 2010 17:54:24 GMT
speculation framework does not update job progress rate with time causing false speculation
-------------------------------------------------------------------------------------------

                 Key: MAPREDUCE-2175
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2175
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: jobtracker
            Reporter: Joydeep Sen Sarma


When considering task speculation - the job's overall progress rate for the given task type
and the individual task's progress rate are compared. When doing so - the task's current progress
rate is considered. However the job's overall progress rate is only only updated when a task's
progress report is received.

This causes weirdness if a particular task does not report for a while. The job's progress
rate is not updated - but the task's progress rate decays. For single task jobs - it causes
speculation almost inevitably. 

We need to update the job's overall progress rate as the task's progress rate changes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message