Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 25 Jul 2017 15:15:01 +0000 (UTC)
From: "Eric Payne (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13021256.1479328337000.7966.1500995701148@Atlassian.JIRA>
In-Reply-To: <JIRA.13021256.1479328337000@Atlassian.JIRA>
References: <JIRA.13021256.1479328337000@Atlassian.JIRA> <JIRA.13021256.1479328337470@jira-lw-us.apache.org>
Subject: [jira] [Commented] (YARN-5892) Support user-specific minimum user
 limit percentage in Capacity Scheduler
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/YARN-5892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16100167#comment-16100167 ] 

Eric Payne commented on YARN-5892:
----------------------------------

Thanks [~sunilg] for the review.
{quote}
In ActiveUsersManager, could we avoid activeUsersChanged if possible. May be we could keep an active set in ActiveUsersManager itself and we could clear this set when activate/deactivateApplication is invoked.
{quote}
In {{ActiveUsersManager}}, the keyset of {{usersApplications}} already holds a list of active users, but it's not enough to know whether or not there are users in this keyset. {{LeafQueue}} also needs to know when a users is added to or removed from this set. This is so that {{LeafQueue#computUserLimit}} is only recomputing the sum of the active user's weights after a user goes active or inactive, not every time through the code.

{{LeafQueue}} could possibly keep its own copy of {{ActiveUsersManager#getNumActiveUsers}} and then compare that with {{getNumActiveUsers}} when it needs to know if it must recompute the sum of active user weights. However, that seems more complicated and error prone than using {{activeUsersChanged}}.


> Support user-specific minimum user limit percentage in Capacity Scheduler
> -------------------------------------------------------------------------
>
>                 Key: YARN-5892
>                 URL: https://issues.apache.org/jira/browse/YARN-5892
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacityscheduler
>            Reporter: Eric Payne
>            Assignee: Eric Payne
>             Fix For: 3.0.0-alpha3
>
>         Attachments: Active users highlighted.jpg, YARN-5892.001.patch, YARN-5892.002.patch, YARN-5892.003.patch, YARN-5892.004.patch, YARN-5892.005.patch, YARN-5892.006.patch, YARN-5892.007.patch, YARN-5892.008.patch, YARN-5892.009.patch, YARN-5892.010.patch, YARN-5892.012.patch, YARN-5892.013.patch, YARN-5892.014.patch, YARN-5892.015.patch, YARN-5892.branch-2.015.patch, YARN-5892.branch-2.016.patch, YARN-5892.branch-2.8.016.patch, YARN-5892.branch-2.8.017.patch, YARN-5892.branch-2.8.018.patch
>
>
> Currently, in the capacity scheduler, the {{minimum-user-limit-percent}} property is per queue. A cluster admin should be able to set the minimum user limit percent on a per-user basis within the queue.
> This functionality is needed so that when intra-queue preemption is enabled (YARN-4945 / YARN-2113), some users can be deemed as more important than other users, and resources from VIP users won't be as likely to be preempted.
> For example, if the {{getstuffdone}} queue has a MULP of 25 percent, but user {{jane}} is a power user of queue {{getstuffdone}} and needs to be guaranteed 75 percent, the properties for {{getstuffdone}} and {{jane}} would look like this:
> {code}
>   <property>
>     <name>yarn.scheduler.capacity.root.getstuffdone.minimum-user-limit-percent</name>
>     <value>25</value>
>   </property>
>   <property>
>     <name>yarn.scheduler.capacity.root.getstuffdone.jane.minimum-user-limit-percent</name>
>     <value>75</value>
>   </property>
> {code}


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org