hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Payne (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5892) Support user-specific minimum user limit percentage in Capacity Scheduler
Date Mon, 10 Jul 2017 18:31:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16080811#comment-16080811

Eric Payne commented on YARN-5892:

Thanks [~sunilg]
bq. YARN-3140 is something i would like to see in branch-2.8 as well. I ll pool some bandwidth
for same, but i might need some more time (considering tests).
Sorry, but I don't think my statement about YARN-3140 was clear.
- I tried backporting YARN-3140 to branch-2.8, but I had several deadlock problems.
- I also tried backporting YARN-3140 to branch-2, and I also ran into deadlock problems, but
not as many and I didn't really try to look into it as hard as branch-2.8
- We are anxious to get the user weights feature backported soon.
I don't think it is worth the effort to try to backport YARN-3140 to branch-2.8. However,
for branch-2, I will make time to try the backport again.
bq. On a related note (if no locking is implemented), I think the following logic is incorrect
May be we can pull this to another ticket and retrospect there. I think we need a second look
in this in detail.
      count += (user != null) ? user.getWeight() : 1.0f;
It's just a matter of changing the 1 to 0. I would like to go ahead and do that in the next

> Support user-specific minimum user limit percentage in Capacity Scheduler
> -------------------------------------------------------------------------
>                 Key: YARN-5892
>                 URL: https://issues.apache.org/jira/browse/YARN-5892
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacityscheduler
>            Reporter: Eric Payne
>            Assignee: Eric Payne
>             Fix For: 3.0.0-alpha3
>         Attachments: Active users highlighted.jpg, YARN-5892.001.patch, YARN-5892.002.patch,
YARN-5892.003.patch, YARN-5892.004.patch, YARN-5892.005.patch, YARN-5892.006.patch, YARN-5892.007.patch,
YARN-5892.008.patch, YARN-5892.009.patch, YARN-5892.010.patch, YARN-5892.012.patch, YARN-5892.013.patch,
YARN-5892.014.patch, YARN-5892.015.patch, YARN-5892.branch-2.015.patch
> Currently, in the capacity scheduler, the {{minimum-user-limit-percent}} property is
per queue. A cluster admin should be able to set the minimum user limit percent on a per-user
basis within the queue.
> This functionality is needed so that when intra-queue preemption is enabled (YARN-4945
/ YARN-2113), some users can be deemed as more important than other users, and resources from
VIP users won't be as likely to be preempted.
> For example, if the {{getstuffdone}} queue has a MULP of 25 percent, but user {{jane}}
is a power user of queue {{getstuffdone}} and needs to be guaranteed 75 percent, the properties
for {{getstuffdone}} and {{jane}} would look like this:
> {code}
>   <property>
>     <name>yarn.scheduler.capacity.root.getstuffdone.minimum-user-limit-percent</name>
>     <value>25</value>
>   </property>
>   <property>
>     <name>yarn.scheduler.capacity.root.getstuffdone.jane.minimum-user-limit-percent</name>
>     <value>75</value>
>   </property>
> {code}

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message