hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-6210) FS: Node reservations can interfere with preemption
Date Tue, 21 Feb 2017 02:34:44 GMT

     [ https://issues.apache.org/jira/browse/YARN-6210?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Karthik Kambatla updated YARN-6210:
    Attachment: YARN-6210.1.patch

Patch (v1) does the following:
# Only starved apps can reserve nodes
# The starvation check considers a single resource. For instance, in the case of DRF, this
corresponds to the dominant resource: (1) the app is considered starved if it is starved on
the dominant resource, (2) an app can be preempted as long as it is not starved on the dominant
# FSAppAttempt#toString for easier debugging of tests
# DominantResourceFairnessComparator#compare can return 0 if the submit time is the same.
This leads to issues while allocating containers. Changed this to be in line with FairShareComparator
and return either -1 or 1.
# Added the test from YARN-6151. FYI. [~yufeigu]

> FS: Node reservations can interfere with preemption
> ---------------------------------------------------
>                 Key: YARN-6210
>                 URL: https://issues.apache.org/jira/browse/YARN-6210
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: fairscheduler
>    Affects Versions: 2.9.0
>            Reporter: Karthik Kambatla
>            Assignee: Karthik Kambatla
>         Attachments: YARN-6210.1.patch
> Today, on a saturated cluster, apps with pending demand reserve nodes. A new app might
not be able to preempt resources because these nodes are already reserved. This can be reproduced
by the example in YARN-6151. 
> Since node reservations are to prevent starvation of apps requesting large containers,
triggering these reservations only on starved applications would avoid this situation. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message