hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Suresh V <verdi...@gmail.com>
Subject Re: Yarn web UI shows more memory used than actual
Date Sun, 14 Aug 2016 01:05:39 GMT
Hello Ray,

I'm referring to the nodes of the cluster page, which shows the individual
nodes and the total memory available in each node and the memory used in
each node.

This 'memory used' would be the memory used by all containers running on
that node; however, if I check free command in the node, there is
significant difference. I'm unable to understand this...

Appreciate any light into this. I agree the main RM page shows the total
containers memory utilization across nodes., which is matching the sum of
memory used in each nodes as displayed in the 'nodes of the cluster' page...

Thank you

Suresh V

On Sat, Aug 13, 2016 at 12:44 PM, Ray Chiang <rchiang@apache.org> wrote:

> The RM page will show the combined container memory usage.  If you have a
> significant difference between any or all of
> 1) actual process memory usage
> 2) JVM heap size
> 3) container maximum
> then you will have significant memory underutilization.
> -Ray
> On 20160813 6:31 AM, Suresh V wrote:
> Hello,
> In our cluster when a MR job is running, in the 'Nodes of the cluster'
> page, it shows the memory used as 84GB out of 87GB allocated to yarn
> nodemanagers.
> However when I actually do a top or free command while logged in to the
> node, it shows as only 23GB used and about 95GB or more free.
> I would imagine the memory used displayed in the Yarn web UI should match
> the memory used shown by top or free command on the node.
> Please advise if this is right thinking or am I missing something?
> Thank you
> Suresh.

View raw message