hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (YARN-5035) FairScheduler: Adjust maxAssign dynamically when assignMultiple is turned on
Date Thu, 26 May 2016 00:19:12 GMT

     [ https://issues.apache.org/jira/browse/YARN-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Karthik Kambatla updated YARN-5035:
    Hadoop Flags: Incompatible change
    Release Note: Introducing a new configuration "yarn.scheduler.fair.dynamic.max.assign"
to dynamically determine the resources to assign per heartbeat when assignmultiple is turned
on. When turned on, the scheduler allocates roughly half of the remaining resources overriding
any max.assign settings configured. This is turned ON by default. 

Marking this an incompatible change as this might affect those who have configured max.assign.
I think it is okay to do this; in my experience, people rarely fiddle with max.assign and
those who do, do so by trial and error to arrive at a behavior similar to the one being introduced

> FairScheduler: Adjust maxAssign dynamically when assignMultiple is turned on
> ----------------------------------------------------------------------------
>                 Key: YARN-5035
>                 URL: https://issues.apache.org/jira/browse/YARN-5035
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: fairscheduler
>    Affects Versions: 2.8.0
>            Reporter: Karthik Kambatla
>            Assignee: Karthik Kambatla
>         Attachments: yarn-5035-1.patch, yarn-5035-2.patch
> When assignMultiple is turned on, today the number of containers allocated to node within
a single heartbeat is determined by maxAssign. Picking a higher value for this prevents the
spreading of the workload, and a smaller number leads to slow ramp up. 
> The proposal is to make it dynamic - half of the unallocated resources on the node. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message