Mailing-List: contact mapreduce-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: mapreduce-issues@hadoop.apache.org
Message-ID: <1599572.37181289513898495.JavaMail.jira@thor>
Date: Thu, 11 Nov 2010 17:18:18 -0500 (EST)
From: "Hong Tang (JIRA)" <jira@apache.org>
To: mapreduce-issues@hadoop.apache.org
Subject: [jira] Commented: (MAPREDUCE-1608) Allow users to do speculative
 execution of a task manually
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/MAPREDUCE-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12931233#action_12931233 ] 

Hong Tang commented on MAPREDUCE-1608:
--------------------------------------

bq. But when there is a manual-speculate-now button, it can override the system-specified speculation heuristics for that task. To go a ste further, the system can actually learn from the user-specified-speculation on how to automatically make the heuristics better, i.e. crowd-sourcing to the rescue!

I agree that knowledgeable users have more information about their programs and probably can speculate more accurately. Such info may also be used for the verification (future tuning) of speculative execution algorithms (e.g. through simulation). So here is my +1 on the idea.

> Allow users to do speculative execution of a task manually
> ----------------------------------------------------------
>
>                 Key: MAPREDUCE-1608
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1608
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>            Reporter: Scott Chen
>            Assignee: Scott Chen
>
> Speculative execution improves the latency of the job. Sometimes the job has few very slow reducers. Spending a little more resource on speculative tasks can improve the latency a lot. It will be nice that the users can manually select one task and force the speculative execution on that task just like we can manually kill/fail task.
> The proposal is add link says "speculate" in taskdetails.jsp page where we do "kill/fail".
> Thoughts? 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.