hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5829) FS preemption should reserve a node before considering containers on it for preemption
Date Thu, 23 Mar 2017 22:51:42 GMT

    [ https://issues.apache.org/jira/browse/YARN-5829?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15939378#comment-15939378

ASF GitHub Bot commented on YARN-5829:

Github user kambatla commented on a diff in the pull request:

    --- Diff: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairScheduler.java
    @@ -987,9 +978,19 @@ void attemptScheduling(FSSchedulerNode node) {
           // Assign new containers...
    -      // 1. Check for reserved applications
    -      // 2. Schedule if there are no reservations
    +      // 1. Ensure containers are assigned to the apps that preempted
    +      // 2. Check for reserved applications
    +      // 3. Schedule if there are no reservations
    +      // Apps may wait for preempted containers
    +      // We have to satisfy these first to avoid cases, when we preempt
    +      // a container for A from B and C gets the preempted containers,
    +      // when C does not qualify for preemption itself.
    +      for (FSAppAttempt app : node.getPreemptionList()) {
    --- End diff --
    We don't seem to handle the case where an app is to be allocated multiple containers from
based on the preempted resources.
    Would it help to store the apps in a TreeMap in FSSchedulerNode and have a method that
returns the next preempted app to allocate to? If none exist, return null? Also, if this were
to work, we might have to revert back to the behavior in an earlier patch: add app/resources
to the map only after the container is actually preempted? 

> FS preemption should reserve a node before considering containers on it for preemption
> --------------------------------------------------------------------------------------
>                 Key: YARN-5829
>                 URL: https://issues.apache.org/jira/browse/YARN-5829
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: fairscheduler
>            Reporter: Karthik Kambatla
>            Assignee: Miklos Szegedi
> FS preemption evaluates nodes for preemption, and subsequently preempts identified containers.
If this node is not reserved for a specific application, any other application could be allocated
resources on this node. 
> Reserving the node for the starved application before preempting containers would help
avoid this.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message