hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1184) mapred.reduce.slowstart.completed.maps is too low by default
Date Mon, 21 Jun 2010 18:41:46 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12880943#action_12880943
] 

Allen Wittenauer commented on MAPREDUCE-1184:
---------------------------------------------

For the *edge case* where jobs benefit from having .05%, they can continue to set this.  The
*average case*, from my experience, is that this is way way way too low.  To re-iterate: I'd
like to change the *default* that Hadoop ships with to something more reasonable for the *average
case*.

> mapred.reduce.slowstart.completed.maps is too low by default
> ------------------------------------------------------------
>
>                 Key: MAPREDUCE-1184
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1184
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>    Affects Versions: 0.20.1, 0.20.2
>            Reporter: Allen Wittenauer
>
> By default, this value is set to 5%.  I believe for most real world situations the code
isn't efficient enough to be set this low.  This should be higher, probably around the 50%
mark, especially given the predominance of non-FIFO schedulers.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message