hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2069) CS queue level preemption should respect user-limits
Date Wed, 09 Jul 2014 00:23:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-2069?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14055701#comment-14055701
] 

Wangda Tan commented on YARN-2069:
----------------------------------

Hi [~mayank_bansal],
I looked at your updated patch, I think the correct logic looks good to me, two comments left:
1) 
{code}
+    Map<String, Resource> consumedResource = new HashMap<String, Resource>();
{code}
I think it's better to name it such as "userMarkedPreemptedResource", it should be preempted
resource, not consumed according to my understanding.

2)
The new added test is good, but I think it's not enough to cover the edge case we mentioned,
The edge case should be
{code}
App1  App2  App3  App4
user1 user1 user2 user2
{code}
And the app<->user created in your test is
{code}
App1  App2  App3  App4
user1 user2 user3 user4
{code}
The preempted resource in the previous case should be 24 containers from app2, 24 container
from app4, 2 containers from app3.

Thanks,
Wangda

> CS queue level preemption should respect user-limits
> ----------------------------------------------------
>
>                 Key: YARN-2069
>                 URL: https://issues.apache.org/jira/browse/YARN-2069
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: capacityscheduler
>            Reporter: Vinod Kumar Vavilapalli
>            Assignee: Mayank Bansal
>         Attachments: YARN-2069-trunk-1.patch, YARN-2069-trunk-2.patch, YARN-2069-trunk-3.patch,
YARN-2069-trunk-4.patch
>
>
> This is different from (even if related to, and likely share code with) YARN-2113.
> YARN-2113 focuses on making sure that even if queue has its guaranteed capacity, it's
individual users are treated in-line with their limits irrespective of when they join in.
> This JIRA is about respecting user-limits while preempting containers to balance queue
capacities.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message