hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Payne (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5892) Capacity Scheduler: Support user-specific minimum user limit percent
Date Fri, 24 Feb 2017 14:03:44 GMT

    [ https://issues.apache.org/jira/browse/YARN-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15882720#comment-15882720
] 

Eric Payne commented on YARN-5892:
----------------------------------

Thanks, [~leftnoteasy], for your feedback. I really value your input.

{quote}
in my mind there're some alternative solutions:
a. Create queue just for such vip users
{quote}

In our multi-tenant clusters, we have several users (sometimes dozens) needing to use the
same queue. Setting up separate queues for each of them based on weighted importance is more
complicated than giving each users their own weight.

bq. #1, if there're N (N <= 100 / MULP) users are consuming resource in a queue, each of
them can get at least MULP / 100 * queue-configured-capacity.

Even today, we can have N > 100/MULP. If I think of these _VIP_ users being the weighted
as multiple users, then we have a similar situation. In your example above, Jack and Alice
would be weighted as 1 user, but Admin would be 2.5 users.

> Capacity Scheduler: Support user-specific minimum user limit percent
> --------------------------------------------------------------------
>
>                 Key: YARN-5892
>                 URL: https://issues.apache.org/jira/browse/YARN-5892
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacityscheduler
>            Reporter: Eric Payne
>            Assignee: Eric Payne
>         Attachments: YARN-5892.001.patch, YARN-5892.002.patch
>
>
> Currently, in the capacity scheduler, the {{minimum-user-limit-percent}} property is
per queue. A cluster admin should be able to set the minimum user limit percent on a per-user
basis within the queue.
> This functionality is needed so that when intra-queue preemption is enabled (YARN-4945
/ YARN-2113), some users can be deemed as more important than other users, and resources from
VIP users won't be as likely to be preempted.
> For example, if the {{getstuffdone}} queue has a MULP of 25 percent, but user {{jane}}
is a power user of queue {{getstuffdone}} and needs to be guaranteed 75 percent, the properties
for {{getstuffdone}} and {{jane}} would look like this:
> {code}
>   <property>
>     <name>yarn.scheduler.capacity.root.getstuffdone.minimum-user-limit-percent</name>
>     <value>25</value>
>   </property>
>   <property>
>     <name>yarn.scheduler.capacity.root.getstuffdone.jane.minimum-user-limit-percent</name>
>     <value>75</value>
>   </property>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message