hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Vinod Kumar Vavilapalli (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-8379) Add an option to allow Capacity Scheduler preemption to balance satisfied queues
Date Thu, 07 Jun 2018 03:51:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-8379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16504207#comment-16504207
] 

Vinod Kumar Vavilapalli commented on YARN-8379:
-----------------------------------------------

Why is this a preemption only concept? To avoid unnecessary thrash between the allocation
doing one thing and the preemption doing another, we should also have a corresponding queue
ordering-policy, right?

> Add an option to allow Capacity Scheduler preemption to balance satisfied queues
> --------------------------------------------------------------------------------
>
>                 Key: YARN-8379
>                 URL: https://issues.apache.org/jira/browse/YARN-8379
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Wangda Tan
>            Assignee: Zian Chen
>            Priority: Major
>         Attachments: YARN-8379.001.patch, YARN-8379.002.patch
>
>
> Existing capacity scheduler only supports preemption for an underutilized queue to reach
its guaranteed resource. In addition to that, there’s an requirement to get better balance
between queues when all of them reach guaranteed resource but with different fairness resource.
> An example is, 3 queues with capacity, queue_a = 30%, queue_b = 30%, queue_c = 40%. At
time T. queue_a is using 30%, queue_b is using 70%. Existing scheduler preemption won't happen.
But this is unfair to queue_a since queue_a has the same guaranteed resources.
> Before YARN-5864, capacity scheduler do additional preemption to balance queues. We changed
the logic since it could preempt too many containers between queues when all queues are satisfied.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message