Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Fri, 6 Mar 2015 17:40:39 +0000 (UTC)
From: "Nathan Roberts (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12779920.1425595840000.22216.1425663639303@Atlassian.JIRA>
In-Reply-To: <JIRA.12779920.1425595840000@Atlassian.JIRA>
References: <JIRA.12779920.1425595840000@Atlassian.JIRA>
 <JIRA.12779920.1425595840312@arcas>
Subject: [jira] [Commented] (YARN-3298) User-limit should be enforced in
 CapacityScheduler
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/YARN-3298?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14350614#comment-14350614 ] 

Nathan Roberts commented on YARN-3298:
--------------------------------------

Hi Wangda. I'm a little concerned about this proposal. I think userlimit has been acting this way for a long time so a change could have a very significant impact on how queues behave.  If I'm understanding the proposal correctly, a queue that is configured with minimum_user_limit_percent=50, capacity=x, max_capacity=10x, user_limit_factor=5 ---- 2 active users would not be able to get the queue above x. Please correct me if that's not the case. Assuming that is the case, I'm not sure that's what we want.

What I see user limit doing is controlling which of the actively requesting applications are getting newly available resources. Basically, making it so that the queue can grow to 10x in the above example while trying to make sure that each of the users within the queue are getting equal shares of capacity.   

> User-limit should be enforced in CapacityScheduler
> --------------------------------------------------
>
>                 Key: YARN-3298
>                 URL: https://issues.apache.org/jira/browse/YARN-3298
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacityscheduler, yarn
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>
> User-limit is not treat as a hard-limit for now, it will not consider required-resource (resource of being-allocated resource request). And also, when user's used resource equals to user-limit, it will still continue. This will generate jitter issues when we have YARN-2069 (preemption policy kills a container under an user, and scheduler allocate a container under the same user soon after).
> The expected behavior should be as same as queue's capacity:
> Only when user.usage + required <= user-limit, queue will continue to allocate container.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)