hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7117) Capacity Scheduler: Support Auto Creation of Leaf Queues While Doing Queue Mapping
Date Tue, 29 Aug 2017 18:30:02 GMT

    [ https://issues.apache.org/jira/browse/YARN-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16145850#comment-16145850

Naganarasimha G R commented on YARN-7117:

Thanks [~wangda] and [~clayb] for bringing up this requirement and [~jlowe] for sharing your
I was more or less thinking in the same way as Jason's approach ??queues are created with
no guarantees (capacity = 0%, max capacity = 100%)?? which would be simple to visualize it.
and then further enhance it based on YARN-5881,
Further i had following queries,
#  These dynamic queues are created under any queue ? how to map which queue this new dynamic
queue is created? i was actually thinking whether we can use reservation queues under default
queues which is not time bounded but will be alive till the apps are running under it. (though
not completely thought about approach)
#  Queue should be created based on user name user group name on what basis will it be decided?
:- i think we need to either support one of it or not sure how to specify whether the new
queue created  should have either name of the user or the group may be pattern ?.
#  For ACL would it be simpler to just have *Submit-ACL* based on user if the  queue is created
based on user and user group if created based on group and *Admin ACLs* inherited from the
parent ?
#  I was little confused with {{Parent queue can be marked to allow auto creation of leaf
queues. ...  We allow no sub queue specified for such parent queue}}, is it that we support
some approach in  YARN-5881 to create parent queue without children ? based on existing configuration
i don't think we can achieve it (correct me if i am wrong). And as well which version are
we targeting this ?

> Capacity Scheduler: Support Auto Creation of Leaf Queues While Doing Queue Mapping
> ----------------------------------------------------------------------------------
>                 Key: YARN-7117
>                 URL: https://issues.apache.org/jira/browse/YARN-7117
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: capacity scheduler
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
> Currently Capacity Scheduler doesn't support auto creation of queues when doing queue
mapping. We saw more and more use cases which has complex queue mapping policies configured
to handle application to queues mapping. 
> The most common use case of CapacityScheduler queue mapping is to create one queue for
each user/group. However update {{capacity-scheduler.xml}} and {{RMAdmin:refreshQueues}} needs
to be done when new user/group onboard. One of the option to solve the problem is automatically
create queues when new user/group arrives.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message