Mailing-List: contact mapreduce-dev-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-dev@hadoop.apache.org
Message-ID: <1676415.222801288806864990.JavaMail.jira@thor>
Date: Wed, 3 Nov 2010 13:54:24 -0400 (EDT)
From: "Joydeep Sen Sarma (JIRA)" <jira@apache.org>
To: mapreduce-dev@hadoop.apache.org
Subject: [jira] Created: (MAPREDUCE-2175) speculation framework does not
 update job progress rate with time causing false speculation
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit

speculation framework does not update job progress rate with time causing false speculation
-------------------------------------------------------------------------------------------

                 Key: MAPREDUCE-2175
                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-2175
             Project: Hadoop Map/Reduce
          Issue Type: Bug
          Components: jobtracker
            Reporter: Joydeep Sen Sarma


When considering task speculation - the job's overall progress rate for the given task type and the individual task's progress rate are compared. When doing so - the task's current progress rate is considered. However the job's overall progress rate is only only updated when a task's progress report is received.

This causes weirdness if a particular task does not report for a while. The job's progress rate is not updated - but the task's progress rate decays. For single task jobs - it causes speculation almost inevitably. 

We need to update the job's overall progress rate as the task's progress rate changes.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.