hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chris Douglas (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1957) ProportionalCapacitPreemptionPolicy handling of corner cases...
Date Tue, 29 Apr 2014 04:27:15 GMT

    [ https://issues.apache.org/jira/browse/YARN-1957?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13983978#comment-13983978
] 

Chris Douglas commented on YARN-1957:
-------------------------------------

+1

Enforcing {{maxCapacity}} in the calculation of the ideal capacity is a good fix, and distributing
capacity over queues with zero capacity (with the config knob to restore the existing 0 ==
disabled with aggressive preemption) makes sense. The code appears to effect this, also. There's
a slight optimization that can separate the zero-capacity queues during cloning, but the overhead
is negligible.

> ProportionalCapacitPreemptionPolicy handling of corner cases...
> ---------------------------------------------------------------
>
>                 Key: YARN-1957
>                 URL: https://issues.apache.org/jira/browse/YARN-1957
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 2.4.0
>            Reporter: Carlo Curino
>            Assignee: Carlo Curino
>              Labels: capacity-scheduler, preemption
>         Attachments: YARN-1957.patch, YARN-1957.patch, YARN-1957_test.patch
>
>
> The current version of ProportionalCapacityPreemptionPolicy should be improved to deal
with the following two scenarios:
> 1) when rebalancing over-capacity allocations, it potentially preempts without considering
the maxCapacity constraints of a queue (i.e., preempting possibly more than strictly necessary)
> 2) a zero capacity queue is preempted even if there is no demand (coherent with old use
of zero-capacity to disabled queues)
> The proposed patch fixes both issues, and introduce few new test cases.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message