hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2069) CS queue level preemption should respect user-limits
Date Tue, 08 Jul 2014 01:01:42 GMT

    [ https://issues.apache.org/jira/browse/YARN-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14054346#comment-14054346

Wangda Tan commented on YARN-2069:

Hi [~mayank_bansal],
Please let me know if I didn't understand correctly,
For the following code snippet,
1      Resource userLimitforQueue = qT.leafQueue.computeUserLimit(fc,
2          clusterResource, Resources.none());
3      if (Resources.lessThan(rc, clusterResource, userLimitforQueue,
4          qT.leafQueue.getUser(fc.getUser()).getConsumedResources())) {
6        // As we have used more resources the user limit,
7        // we need to claim back the resources equivalent to
8        // consumed resources by user - user limit
9        Resource resourcesToClaimBackFromUser = Resources.subtract(qT.leafQueue
10          .getUser(fc.getUser()).getConsumedResources(), userLimitforQueue);
Line 1-4 will check if we need preempt resource from an application, because preemption is
a delayed behavior, the "preemptFrom" will not change return value of qT.leafQueue.getUser(fc.getUser()).getConsumedResources().
So for 5 apps of B, all its apps will be preempt line 9-10 containers. Is it correct?

> CS queue level preemption should respect user-limits
> ----------------------------------------------------
>                 Key: YARN-2069
>                 URL: https://issues.apache.org/jira/browse/YARN-2069
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Mayank Bansal
>         Attachments: YARN-2069-trunk-1.patch, YARN-2069-trunk-2.patch, YARN-2069-trunk-3.patch
> This is different from (even if related to, and likely share code with) YARN-2113.
> YARN-2113 focuses on making sure that even if queue has its guaranteed capacity, it's
individual users are treated in-line with their limits irrespective of when they join in.
> This JIRA is about respecting user-limits while preempting containers to balance queue

This message was sent by Atlassian JIRA

View raw message