hadoop-mapreduce-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Hans-Peter Zorn <hpz...@gmail.com>
Subject Capacity Scheduler on YARN
Date Tue, 30 Jul 2013 09:33:59 GMT

I would like to use the capacity scheduler to implement the following

3 groups of users:
* 2 of them are entitled to 50% of cluster capacity each but may use excess
capacity if available
* the third group only may use capacity that is currently not used by any
of the other two.

I set up three queues:
yarn.scheduler.capacity.root.a.capacity 50
yarn.scheduler.capacity.root.a.maximum-capacity 100

yarn.scheduler.capacity.root.b.capacity 49
yarn.scheduler.capacity.root.b.maximum-capacity 100

yarn.scheduler.capacity.root.c.capacity 1
yarn.scheduler.capacity.root.c.maximum-capacity 100

These values are active, as they show up in the scheduler web gui.

However, users of each queue are only able to use the assigned capacity,
the value
of maximum-capacity has no effect.

Did I misunderstand the concept behind the maximum-capacity setting?

I am using hadoop as shipped with Cloudera CDH4.3  (2.0.0-cdh4.3.0). I
asked this question already on cdh-users, but the capacity scheduler seems
not to be used so much with cdh I guess.

Or is it possible to implement this using a different scheduler?

Best & thanks,

View raw message