hadoop-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arun C Murthy <...@hortonworks.com>
Subject Re: How is the memory usage of containers controlled?
Date Fri, 21 Jun 2013 20:50:49 GMT
Start with LinuxResourceCalculatorPlugin.


On Jun 21, 2013, at 1:30 PM, Yuzhang Han <yuzhanghan1982@gmail.com> wrote:

> Thank you Arun.
> I am trying to study how YARN works. Can someone tell me which class(es) monitors containers
w.r.t. memory usage?
> On 6/18/2013 4:07 PM, Arun C Murthy wrote:
>> NodeManagers monitor containers w.r.t memory usage, and put containers in cgroups
with cpu limits to restrict CPU usage.
>> On Jun 18, 2013, at 12:30 PM, Yuzhang Han <yuzhanghan1982@gmail.com> wrote:
>>> Hi,
>>> I am curious about how YARN containers control their memory usage. Say, I have
a MR job, and I configure that every map task should be assigned a 1 GB container, and every
reduce task a 1.5 GB one. So, when YARN runs the containers, how is it ensured that all map
containers use less than 1 GB memory and all reduce containers less than 1.5 GB? Is container
memory control realized via -Xmx arguments in the scripts generated by YARN to invoke the
MR program in the containers?
>>> Thank you.
>> --
>> Arun C. Murthy
>> Hortonworks Inc.
>> http://hortonworks.com/

Arun C. Murthy
Hortonworks Inc.

View raw message