hadoop-mapreduce-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (MAPREDUCE-5906) Inconsistent configuration in property "mapreduce.reduce.shuffle.input.buffer.percent"
Date Thu, 14 Aug 2014 18:16:12 GMT

    [ https://issues.apache.org/jira/browse/MAPREDUCE-5906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14097320#comment-14097320
] 

Hudson commented on MAPREDUCE-5906:
-----------------------------------

FAILURE: Integrated in Hadoop-trunk-Commit #6069 (See [https://builds.apache.org/job/Hadoop-trunk-Commit/6069/])
MAPREDUCE-5906. Inconsistent configuration in property "mapreduce.reduce.shuffle.input.buffer.percent"
(Akira AJISAKA via aw) (aw: http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1618007)
* /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/MRJobConfig.java
* /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/java/org/apache/hadoop/mapreduce/task/reduce/MergeManagerImpl.java


> Inconsistent configuration in property "mapreduce.reduce.shuffle.input.buffer.percent"
> --------------------------------------------------------------------------------------
>
>                 Key: MAPREDUCE-5906
>                 URL: https://issues.apache.org/jira/browse/MAPREDUCE-5906
>             Project: Hadoop Map/Reduce
>          Issue Type: Improvement
>    Affects Versions: 2.4.0
>            Reporter: Akira AJISAKA
>            Assignee: Akira AJISAKA
>            Priority: Minor
>              Labels: newbie
>             Fix For: 3.0.0, 2.6.0
>
>         Attachments: MAPREDUCE-5906.patch
>
>
> In MergeManagerImpl.java, the default value of MRJobConfig.SHUFFLE_INPUT_BUFFER_PERCENT
(=mapreduce.reduce.shuffle.input.buffer.percent) looks 0.90.
> {code}
>   final float maxInMemCopyUse =
>     jobConf.getFloat(MRJobConfig.SHUFFLE_INPUT_BUFFER_PERCENT, 0.90f);
> {code}
> However, the actual default value is 0.70 in mapred-default.xml.
> {code}
>   <name>mapreduce.reduce.shuffle.input.buffer.percent</name>
>   <value>0.70</value>
> {code}



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message