hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7117) Capacity Scheduler: Support Auto Creation of Leaf Queues While Doing Queue Mapping
Date Mon, 02 Oct 2017 19:33:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16188674#comment-16188674

Jason Lowe commented on YARN-7117:

bq. The current CS code has a bug in that it allows "." in the queue names. Do you think we
should fix this for 3.0? 

That's unfortunate, but I do not think it has to be a show-stopper.  A couple of ways to fix

1) Preclude the use of '.' in auto-queue names, then we always know the last word when we
split with '.' is the child queue and the rest is the parent queue.


2) The parsing of the specified queue names becomes trickier.  Rather than the code blindly
assuming it can split the name on '.' to get queue names, it has to check to see if the parent
exists.  If it doesn't then it adds the next chunk from the split and see if that's a valid
parent queue, etc.  There may be some issues with ambiguity, but I suspect there would be
other problems with getting queue configs parsed properly if there were truly ambiguous resolutions.

bq. This could be done though it might be a backward in-compatible change. 

Which part has the compatibility concern?  There aren't any existing semantics of auto-queues
since they doesn't exist yet, so I'm confused where the compatibility issue lies.

> Capacity Scheduler: Support Auto Creation of Leaf Queues While Doing Queue Mapping
> ----------------------------------------------------------------------------------
>                 Key: YARN-7117
>                 URL: https://issues.apache.org/jira/browse/YARN-7117
>             Project: Hadoop YARN
>          Issue Type: New Feature
>          Components: capacity scheduler
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: YARN-7117.Capacity.Scheduler.Support.Auto.Creation.Of.Leaf.Queue.pdf
> Currently Capacity Scheduler doesn't support auto creation of queues when doing queue
mapping. We saw more and more use cases which has complex queue mapping policies configured
to handle application to queues mapping. 
> The most common use case of CapacityScheduler queue mapping is to create one queue for
each user/group. However update {{capacity-scheduler.xml}} and {{RMAdmin:refreshQueues}} needs
to be done when new user/group onboard. One of the option to solve the problem is automatically
create queues when new user/group arrives.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message