hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Thon, Ingo" <ingo.t...@siemens.com>
Subject cluster utiliation when using fair scheduler
Date Wed, 15 Apr 2015 07:56:27 GMT

I'm using the fair scheduler for Yarn. I have not specified any pools, so the fair-scheduler.xml
is basically empty.
However, only one third of the cluster is utilized.

On the scheduler page I see a single Queue which is root and it is specified that 33.3% used
This 33.3% is independent of the number of jobs.
Currently, I have a single job running and it is claimed that for this Application pipeline:
Used Resources: <memory:196608, vCores:48>
Num Active Applications:        21
Num Pending Applications:       0
Min Resources:  <memory:0, vCores:0>
Max Resources:  <memory:589824, vCores:48>
Fair Share:     <memory:45372, vCores:0>

The number of vCores appears to be to low as well. The number of cores should be 6x16.

Any suggestions what to check?

Cheers Ingo

View raw message