hadoop-common-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Fox <dafox7777...@gmail.com>
Subject YARN Fair Scheduler - maxResources limit for parent and child queues with inheritance
Date Tue, 29 Sep 2015 15:29:51 GMT
Hello,

I would like to know how it works with Fair Scheduler in YARN-hadoop. I'm
trying to configure parent queue with MaxResorces in allocations file, e.i.
fair-scheduler.xml. Then I want to create child queues. As documentation
about fair scheduler says "Queues can be arranged in a hierarchy to divide
resources and configured with weights to share the cluster in specific
proportions"
http://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-site/FairScheduler.html
I suppose if I configure parent queue with Max Resources, the child queues
will be limited to max of the parent resources with fair shares. Is my
undersatnding of queue hierarchy mechanism right?

Example to be more concrete:
{code}
<?xml version="1.0"?>
<allocations>
    <queue name="someParentQueue" type="parent">
        <maxResources>111000 mb,100 vcores, 100 disks</maxResources>
    </queue>
    <queuePlacementPolicy>
        <rule name="nestedUserQueue" create="true">
            <rule name="specified" create="false"/>
        </rule>
        <rule name="user" create="true"/>
    </queuePlacementPolicy>
</allocations>
{code}

The queues someParentQueue.user1 has different Max Resources in web UI. Is
it issue of web UI or intended behavoior of Fair Scheduler?

Thanks

Mime
View raw message