hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Devaraj Das (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-3117) Speculative execution configuration is confusing
Date Mon, 31 Mar 2008 06:22:24 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-3117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12583562#action_12583562
] 

Devaraj Das commented on HADOOP-3117:
-------------------------------------

As per the trunk, there are two switches for controlling speculative execution - mapred.{map,reduce}.tasks.speculative.execution.
The default value is true for both.

The corresponding JobConf APIs are setMapSpeculativeExecution(boolean) & setReduceSpeculativeExecution(boolean)

There is a third API - setSpeculativeExecution(boolean). This sets true or false for both
types of tasks. (Pls see HADOOP-2819).

>From the config point of view, having two distinct knobs for maps and reduces makes things
more clear, no?

> Speculative execution configuration is confusing
> ------------------------------------------------
>
>                 Key: HADOOP-3117
>                 URL: https://issues.apache.org/jira/browse/HADOOP-3117
>             Project: Hadoop Core
>          Issue Type: Improvement
>          Components: conf
>    Affects Versions: 0.16.0
>            Reporter: Sameer Paranjpye
>            Assignee: Devaraj Das
>             Fix For: 0.17.0
>
>
> In Hadoop 0.16 the single switch for speculative execution was replaced by 2 switches
for maps and reduces. The old switch was retained as well and overrides the new switches if
it is turned on. This is confusing. We should go back to a single config variable that has
one value from the set  -- 'true, false, map, reduce'.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message