hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Payne (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5892) Capacity Scheduler: Support user-specific minimum user limit percent
Date Mon, 03 Apr 2017 15:05:41 GMT

    [ https://issues.apache.org/jira/browse/YARN-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15953639#comment-15953639
] 

Eric Payne commented on YARN-5892:
----------------------------------

Thanks [~leftnoteasy] for your detailed review. I really appreciate it.

I have a couple of clarifications:
bq. 1) For User weight:I prefer to only allow specify it in the leaf queue level.
I think it's good to allow setting user weights at parent-queue levels. This allows cluster
admins to set it once per user per cluster instead of on multiple leaf queues.
bq.  should we allow overwriting in leaf
Yes, with the current implementing, settings in leaf queues would override those in parent
queues.
bq. {{getUserAMResourceLimitPerPartition(String)}} is not used by anyone, can be removed.
Since this was implemented as a public interface, I wanted to keep this API here for backwards
compatibility, in case something outside Hadoop was depending on it. If this is not necessary,
we can remove it.
bq. Should we move userLimitNeedsRecompute into the _*weight*_ lock
Did you mean "_*write*_ lock"? ;-)


> Capacity Scheduler: Support user-specific minimum user limit percent
> --------------------------------------------------------------------
>
>                 Key: YARN-5892
>                 URL: https://issues.apache.org/jira/browse/YARN-5892
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacityscheduler
>            Reporter: Eric Payne
>            Assignee: Eric Payne
>         Attachments: Active users highlighted.jpg, YARN-5892.001.patch, YARN-5892.002.patch,
YARN-5892.003.patch, YARN-5892.004.patch, YARN-5892.005.patch, YARN-5892.006.patch
>
>
> Currently, in the capacity scheduler, the {{minimum-user-limit-percent}} property is
per queue. A cluster admin should be able to set the minimum user limit percent on a per-user
basis within the queue.
> This functionality is needed so that when intra-queue preemption is enabled (YARN-4945
/ YARN-2113), some users can be deemed as more important than other users, and resources from
VIP users won't be as likely to be preempted.
> For example, if the {{getstuffdone}} queue has a MULP of 25 percent, but user {{jane}}
is a power user of queue {{getstuffdone}} and needs to be guaranteed 75 percent, the properties
for {{getstuffdone}} and {{jane}} would look like this:
> {code}
>   <property>
>     <name>yarn.scheduler.capacity.root.getstuffdone.minimum-user-limit-percent</name>
>     <value>25</value>
>   </property>
>   <property>
>     <name>yarn.scheduler.capacity.root.getstuffdone.jane.minimum-user-limit-percent</name>
>     <value>75</value>
>   </property>
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message