hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2069) CS queue level preemption should respect user-limits
Date Wed, 09 Jul 2014 00:23:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14055701#comment-14055701

Wangda Tan commented on YARN-2069:

Hi [~mayank_bansal],
I looked at your updated patch, I think the correct logic looks good to me, two comments left:
+    Map<String, Resource> consumedResource = new HashMap<String, Resource>();
I think it's better to name it such as "userMarkedPreemptedResource", it should be preempted
resource, not consumed according to my understanding.

The new added test is good, but I think it's not enough to cover the edge case we mentioned,
The edge case should be
App1  App2  App3  App4
user1 user1 user2 user2
And the app<->user created in your test is
App1  App2  App3  App4
user1 user2 user3 user4
The preempted resource in the previous case should be 24 containers from app2, 24 container
from app4, 2 containers from app3.


> CS queue level preemption should respect user-limits
> ----------------------------------------------------
>                 Key: YARN-2069
>                 URL: https://issues.apache.org/jira/browse/YARN-2069
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Mayank Bansal
>         Attachments: YARN-2069-trunk-1.patch, YARN-2069-trunk-2.patch, YARN-2069-trunk-3.patch,
> This is different from (even if related to, and likely share code with) YARN-2113.
> YARN-2113 focuses on making sure that even if queue has its guaranteed capacity, it's
individual users are treated in-line with their limits irrespective of when they join in.
> This JIRA is about respecting user-limits while preempting containers to balance queue

This message was sent by Atlassian JIRA

View raw message