hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@hortonworks.com>
Subject Re: How mapred.map.child.java.opts influence the Container resource allocation
Date Mon, 04 Feb 2013 15:43:48 GMT
Youpeng,

 The *container* is a generic concept which allows the ResourceManager to provide you a specific
amount of resources (memory, cpu etc.).

 mapred.map.child.java.opts is part of the cmd-line of the Map task's jvm which specifies
the heap size.

 So, in YARN, mapred.map.child.java.opts should be less than memory allocated to container.

 Remember that container memory is physical memory of the unix process, while mapred.map.child.java.opts
is heap size which doesn't account for non-heap memory.

 So, with a container size of 2G, the max heap size you can allocate is ~1.6G or so.

hth,
Arun

On Feb 4, 2013, at 3:38 AM, YouPeng Yang wrote:

> Hi All
> 
>     I use YARN and MR.I am confused by the relationship between 
> mapred.map.child.java.opts and  Container resource  allocation.
>     if I setup mapred.map.child.java.opts = -Xmx2048m ,on nodemanager
> how Containers will be allocated according to this settings.
>      what about mapred.reduce.child.java.opts?
> 
> regards 
> Youpeng Yang

--
Arun C. Murthy
Hortonworks Inc.
http://hortonworks.com/



Mime
View raw message