hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2069) CS queue level preemption should respect user-limits
Date Tue, 08 Jul 2014 00:44:34 GMT

    [ https://issues.apache.org/jira/browse/YARN-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14054334#comment-14054334

Wangda Tan commented on YARN-2069:

Hi [~mayank_bansal],
Thanks for your comments,
I think the change of title/description should be correct, this patch is targeted to solve
cross-queue preemption should respect user-limit.

I think your other comments all make sense to me. Only below one,
bq. We need to maintian the reverse order of application submission which only can be done
iterating through applications as we want to preempt applications which are last submitted.
IMHO, this is reasonable but conflict with this JIRA's scope, let me give you an example.

Assume a queue has 10 apps, each app has 5 containers (1G for each container, so queue has
50G mem used). There're two apps, each app has 5 apps. User-limit is 15G, queue's absolute
capacity is 30G.
And first 5 apps belongs to user-A, last 5 apps belongs to user-B.
In your correct method, user-B will be preempted 20 containers and user-A will be preempted
After preemption, only 5 container left for user-B, and 25 containers left for user-A. User-limit
is respected here.

Does this make sense to you?


> CS queue level preemption should respect user-limits
> ----------------------------------------------------
>                 Key: YARN-2069
>                 URL: https://issues.apache.org/jira/browse/YARN-2069
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Mayank Bansal
>         Attachments: YARN-2069-trunk-1.patch, YARN-2069-trunk-2.patch, YARN-2069-trunk-3.patch
> This is different from (even if related to, and likely share code with) YARN-2113.
> YARN-2113 focuses on making sure that even if queue has its guaranteed capacity, it's
individual users are treated in-line with their limits irrespective of when they join in.
> This JIRA is about respecting user-limits while preempting containers to balance queue

This message was sent by Atlassian JIRA

View raw message