ambari-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hudson (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMBARI-10979) MRv2 Config Screen allows me to allocate sort buffers larger than heap.
Date Wed, 06 May 2015 23:38:01 GMT

    [ https://issues.apache.org/jira/browse/AMBARI-10979?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14531688#comment-14531688
] 

Hudson commented on AMBARI-10979:
---------------------------------

SUCCESS: Integrated in Ambari-trunk-Commit #2534 (See [https://builds.apache.org/job/Ambari-trunk-Commit/2534/])
AMBARI-10979. MRv2 Config Screen allows me to allocate sort buffers larger than heap. (srimanth)
(sgunturi: http://git-wip-us.apache.org/repos/asf?p=ambari.git&a=commit&h=d7e8a796916e264222eb44672fd65619dcb2108c)
* ambari-server/src/main/resources/stacks/HDP/2.2/services/stack_advisor.py
* ambari-server/src/test/python/stacks/2.2/common/test_stack_advisor.py


> MRv2 Config Screen allows me to allocate sort buffers larger than heap.
> -----------------------------------------------------------------------
>
>                 Key: AMBARI-10979
>                 URL: https://issues.apache.org/jira/browse/AMBARI-10979
>             Project: Ambari
>          Issue Type: Bug
>          Components: contrib
>    Affects Versions: 2.1.0
>            Reporter: Srimanth Gunturi
>            Assignee: Srimanth Gunturi
>             Fix For: 2.1.0
>
>         Attachments: AMBARI-10979.patch
>
>
> {{mapreduce.task.io.sort.mb}} needs to be smaller than the min Xmx for maps and reduces
(also applies to Tez).
> Ambari allows me to specify an invalid state without any warnings.
> The formulas should be as follows
> {code}
>     'mapreduce.map.memory.mb' = int(configurations["yarn-site"]["properties"]["yarn.scheduler.minimum-allocation-mb"])
>     'mapreduce.reduce.memory.mb' = min(2*int(configurations["yarn-site"]["properties"]["yarn.scheduler.minimum-allocation-mb"]),
int(nodemanagerMinRam))
>     mapredMapXmx = int(0.8*int(configurations["mapred-site"]["properties"]["mapreduce.map.memory.mb"]));
>     'mapreduce.map.java.opts' = "-Xmx" + str(mapredMapXmx) + "m"
>     'mapreduce.reduce.java.opts' = "-Xmx" + str(int(0.8*int(configurations["mapred-site"]["properties"]["mapreduce.reduce.memory.mb"])))
+ "m"
>     'mapreduce.task.io.sort.mb' = str(min(int(0.7*mapredMapXmx), 2047))
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message