hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Craig Welch (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3510) Create an extension of ProportionalCapacityPreemptionPolicy which preempts a number of containers from each application in a way which respects fairness
Date Fri, 29 May 2015 00:28:33 GMT

    [ https://issues.apache.org/jira/browse/YARN-3510?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14563983#comment-14563983
] 

Craig Welch commented on YARN-3510:
-----------------------------------

The attached patch supports an optional configuration option for the preemption policy, {code}
yarn.resourcemanager.monitor.capacity.preemption.preempt_evenly{code}, which (when set to
true) causes the policy to only preempt one live container per application per round, and
to do multiple rounds until the desired resources are obtained (or no further progress is
occurring), so that preemption should generally maintain existing relative usage between apps.
  This is in contrast to the default behavior (when unset or set to false) (equivalent to
the existing behavior), which is to take as much as possible from each app in order of the
preemption iterator.  The default works well for the fifo case, but will unbalance usage between
apps in the fair case.

> Create an extension of ProportionalCapacityPreemptionPolicy which preempts a number of
containers from each application in a way which respects fairness
> --------------------------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-3510
>                 URL: https://issues.apache.org/jira/browse/YARN-3510
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: yarn
>            Reporter: Craig Welch
>            Assignee: Craig Welch
>         Attachments: YARN-3510.2.patch
>
>
> The ProportionalCapacityPreemptionPolicy preempts as many containers from applications
as it can during it's preemption run.  For fifo this makes sense, as it is prempting in reverse
order & therefore maintaining the primacy of the "oldest".  For fair ordering this does
not have the desired effect - instead, it should preempt a number of containers from each
application which maintains a fair balance /close to a fair balance between them



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message