hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Todd Lipcon (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1608) Allow users to do speculative execution of a task manually
Date Fri, 19 Mar 2010 00:34:30 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1608?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12847172#action_12847172
] 

Todd Lipcon commented on MAPREDUCE-1608:
----------------------------------------

I'm not entirely against a manual trigger, but this suggests that our current tuning knobs
are insufficient. What are the cases when users would want to do this manually, and why don't
our heuristics do it for them? I'm skeptical that humans can do a better job than the scheduler
at deciding when speculation is necessary.

> Allow users to do speculative execution of a task manually
> ----------------------------------------------------------
>
>                 Key: MAPREDUCE-1608
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1608
>             Project: Hadoop Map/Reduce
>          Issue Type: New Feature
>            Reporter: Scott Chen
>            Assignee: Scott Chen
>
> Speculative execution improves the latency of the job. Sometimes the job has few very
slow reducers. Spending a little more resource on speculative tasks can improve the latency
a lot. It will be nice that the users can manually select one task and force the speculative
execution on that task just like we can manually kill/fail task.
> The proposal is add link says "speculate" in taskdetails.jsp page where we do "kill/fail".
> Thoughts? 

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message