hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy Ryza (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1290) Let continuous scheduling achieve more balanced task assignment
Date Tue, 15 Oct 2013 02:14:42 GMT

    [ https://issues.apache.org/jira/browse/YARN-1290?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13794788#comment-13794788

Sandy Ryza commented on YARN-1290:

I was concerned that the sort would have a performance impact, but ran it through a container
assigning microbenchmark with 1000 nodes and didn't see an significant effect.


> Let continuous scheduling achieve more balanced task assignment
> ---------------------------------------------------------------
>                 Key: YARN-1290
>                 URL: https://issues.apache.org/jira/browse/YARN-1290
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Wei Yan
>            Assignee: Wei Yan
>         Attachments: main.pdf, YARN-1290.patch, YARN-1290.patch, YARN-1290.patch
> Currently, in continuous scheduling (YARN-1010), in each round, the thread iterates over
pre-ordered nodes and assigns tasks. This mechanism may overload the first several nodes,
while the latter nodes have no tasks.
> We should sort all nodes according to available resource. In each round, always assign
tasks to nodes with larger capacity, which can balance the load distribution among all nodes.

This message was sent by Atlassian JIRA

View raw message