hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6163) FS Preemption is a trickle for severely starved applications
Date Sat, 11 Feb 2017 17:59:42 GMT

    [ https://issues.apache.org/jira/browse/YARN-6163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15862470#comment-15862470
] 

ASF GitHub Bot commented on YARN-6163:
--------------------------------------

Github user kambatla commented on a diff in the pull request:

    https://github.com/apache/hadoop/pull/192#discussion_r100672986
  
    --- Diff: hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/src/main/java/org/apache/hadoop/yarn/server/resourcemanager/scheduler/fair/FairSchedulerConfiguration.java
---
    @@ -114,12 +114,24 @@
       protected static final String PREEMPTION_THRESHOLD =
           CONF_PREFIX + "preemption.cluster-utilization-threshold";
       protected static final float DEFAULT_PREEMPTION_THRESHOLD = 0.8f;
    -  
    -  protected static final String PREEMPTION_INTERVAL = CONF_PREFIX + "preemptionInterval";
    -  protected static final int DEFAULT_PREEMPTION_INTERVAL = 5000;
    +
       protected static final String WAIT_TIME_BEFORE_KILL = CONF_PREFIX + "waitTimeBeforeKill";
       protected static final int DEFAULT_WAIT_TIME_BEFORE_KILL = 15000;
     
    +  /**
    +   * Configurable delay before an app's starvation is considered after it is
    +   * identified. This is to give the scheduler enough time to
    +   * allocate containers post preemption. This delay is added to the
    +   * {@link #WAIT_TIME_BEFORE_KILL} and enough heartbeats.
    +   *
    +   * This is intended as a backdoor on production clusters, and hence
    +   * intentionally not documented.
    +   */
    +  protected static final String WAIT_TIME_BEFORE_NEXT_STARVATION_CHECK =
    --- End diff --
    
    Done


> FS Preemption is a trickle for severely starved applications
> ------------------------------------------------------------
>
>                 Key: YARN-6163
>                 URL: https://issues.apache.org/jira/browse/YARN-6163
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: fairscheduler
>    Affects Versions: 2.9.0
>            Reporter: Karthik Kambatla
>            Assignee: Karthik Kambatla
>         Attachments: yarn-6163-1.patch
>
>
> With current logic, only one RR is considered per each instance of marking an application
starved. This marking happens only on the update call that runs every 500ms.  Due to this,
an application that is severely starved takes forever to reach fairshare based on preemptions.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message