hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-2801) Non-speculative and speculative tasks for a TIP start at the same time.
Date Fri, 08 Feb 2008 20:39:07 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-2801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12567197#action_12567197

Devaraj Das commented on HADOOP-2801:

That's right. The startTime is initialized just once inside TaskInProgress and the speculative
check takes that into consideration. So for cases where a failed task is retried (let's say
at least after a 1 minute interval), and another tasktracker asks for a task at around the
same time instant, the check _(System.currentTimeMillis() - startTime >= SPECULATIVE_LAG)_
 is going to return true, and a speculative instance will be launched almost simultaneously.
Also i noticed there is a related field execStartTime. This is also updated only once. 

> Non-speculative and speculative tasks for a TIP start at the same time.
> -----------------------------------------------------------------------
>                 Key: HADOOP-2801
>                 URL: https://issues.apache.org/jira/browse/HADOOP-2801
>             Project: Hadoop Core
>          Issue Type: Bug
>          Components: mapred
>            Reporter: Amar Kamat
>             Fix For: 0.16.1
> I ran into a situation where the non-speculative task and the speculative task were started
at the same time. Ideally there should be a 1 min gap for speculation. One possible reason
would be the usage of {{startTime}} in deciding whether there is a speculative lag.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message