hadoop-hdfs-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ashwin Sai Shankar <ashan...@netflix.com>
Subject Re: YARN Fair Scheduler - maxResources limit for parent and child queues with inheritance
Date Wed, 07 Oct 2015 17:41:59 GMT
Yes this is a issue with webui. maxResources would be enforced at
*so childQueues of it will get atmost ${maxResources} depending
on usage of its siblings.

On Tue, Sep 29, 2015 at 8:29 AM, David Fox <dafox7777777@gmail.com> wrote:

> Hello,
> I would like to know how it works with Fair Scheduler in YARN-hadoop. I'm
> trying to configure parent queue with MaxResorces in allocations file, e.i.
> fair-scheduler.xml. Then I want to create child queues. As documentation
> about fair scheduler says "Queues can be arranged in a hierarchy to divide
> resources and configured with weights to share the cluster in specific
> proportions"
> http://hadoop.apache.org/docs/r2.7.0/hadoop-yarn/hadoop-yarn-site/FairScheduler.html
> I suppose if I configure parent queue with Max Resources, the child queues
> will be limited to max of the parent resources with fair shares. Is my
> undersatnding of queue hierarchy mechanism right?
> Example to be more concrete:
> {code}
> <?xml version="1.0"?>
> <allocations>
>     <queue name="someParentQueue" type="parent">
>         <maxResources>111000 mb,100 vcores, 100 disks</maxResources>
>     </queue>
>     <queuePlacementPolicy>
>         <rule name="nestedUserQueue" create="true">
>             <rule name="specified" create="false"/>
>         </rule>
>         <rule name="user" create="true"/>
>     </queuePlacementPolicy>
> </allocations>
> {code}
> The queues someParentQueue.user1 has different Max Resources in web UI. Is
> it issue of web UI or intended behavoior of Fair Scheduler?
> Thanks

View raw message