hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eric Payne (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-2056) Disable preemption at Queue level
Date Thu, 21 Aug 2014 01:38:25 GMT

     [ https://issues.apache.org/jira/browse/YARN-2056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Eric Payne updated YARN-2056:

    Attachment: YARN-2056.201408202039.txt

This patch keeps the {{yarn.resourcemanager.monitor.capacity.preemption.max_ignored_over_capacity}}
property as a global parameter, and then adds a per-queue property in this format: {{yarn.resourcemanager.monitor.capacity.preemption.<queue-path>.max_ignored_over_capacity}}

The preemption code makes two sets of passes through the queues. The first time through, it
calculates the ideal resource allocation per queue based on normalized guaranteed capacity,
and the second time through, it selects which queue's resources to preempt, taking into consideration
the {{max_ignored_over_capacity))

In this patch, the per-queue {{...max_ignored_over_capacity}} is taken into consideration
in the first pass to help determine which queues have resources available for preempting.
This is necessary because without it, queues that could fulfill the need would otherwise be
removed from the list of available resources. Then, for the second pass, the global {{...max_ignored_over_capacity}}
setting is used, as before, to determine which resources out of the remaining available resources
to use.

This patch still requires an RM restart if the queue properties have changed.

> Disable preemption at Queue level
> ---------------------------------
>                 Key: YARN-2056
>                 URL: https://issues.apache.org/jira/browse/YARN-2056
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>    Affects Versions: 2.4.0
>            Reporter: Mayank Bansal
>            Assignee: Eric Payne
>         Attachments: YARN-2056.201408202039.txt
> We need to be able to disable preemption at individual queue level

This message was sent by Atlassian JIRA

View raw message