hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod K V (JIRA)" <j...@apache.org>
Subject [jira] Commented: (MAPREDUCE-1184) mapred.reduce.slowstart.completed.maps is too high by default
Date Fri, 06 Nov 2009 10:13:32 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-1184?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12774263#action_12774263

Vinod K V commented on MAPREDUCE-1184:

As such I an neutral to this change. But as I said before, this is site-specific, job-specific.
And this comment I've made only after seeing different opinions on this issue itself.

Allen says in the description "This should be higher, probably around the 50% mark".
Hong says [here|https://issues.apache.org/jira/browse/MAPREDUCE-1184?focusedCommentId=12773788&page=com.atlassian.jira.plugin.system.issuetabpanels%3Acomment-tabpanel#action_12773788]
"Another case, for tiny jobs that require fast turn around time, it would be better if we
set the percentage to be 0."

Either there is a cross-talk or I am missing something or these are different use-cases already.
If we can agree to a default that is agreeable to everyone here, I am fine.

> mapred.reduce.slowstart.completed.maps is too high by default
> -------------------------------------------------------------
>                 Key: MAPREDUCE-1184
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-1184
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>            Reporter: Allen Wittenauer
> By default, this value is set to 5%.  I believe for most real world situations the code
isn't efficient enough to be set this low.  This should be higher, probably around the 50%
mark, especially given the predominance of non-FIFO schedulers.

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message