hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5035) FairScheduler: Adjust maxAssign dynamically when assignMultiple is turned on
Date Wed, 25 May 2016 23:53:12 GMT

    [ https://issues.apache.org/jira/browse/YARN-5035?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15301158#comment-15301158

Karthik Kambatla commented on YARN-5035:

bq. Looks to me like you can never assign the last container on a node because Resources.multiplyTo()
will round down
assignmultiple and maxassign come into play only after we allocate one container. So, we shouldn't
run into this. Also, the intent is for a rough half of the resources so we allocate multiple
containers without overloading any one machine. Let me augment the test in any case to make
sure we fully allocate the node. 

bq. Also you have an "upto" in the MD code that you should fix.
Given the allocation is roughly half the containers. I should change this to "about".

> FairScheduler: Adjust maxAssign dynamically when assignMultiple is turned on
> ----------------------------------------------------------------------------
>                 Key: YARN-5035
>                 URL: https://issues.apache.org/jira/browse/YARN-5035
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: fairscheduler
>    Affects Versions: 2.8.0
>            Reporter: Karthik Kambatla
>            Assignee: Karthik Kambatla
>         Attachments: yarn-5035-1.patch
> When assignMultiple is turned on, today the number of containers allocated to node within
a single heartbeat is determined by maxAssign. Picking a higher value for this prevents the
spreading of the workload, and a smaller number leads to slow ramp up. 
> The proposal is to make it dynamic - half of the unallocated resources on the node. 

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message