hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1011) [Umbrella] Schedule containers based on utilization of currently allocated containers
Date Sun, 27 Dec 2015 19:46:50 GMT

    [ https://issues.apache.org/jira/browse/YARN-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15072227#comment-15072227
] 

Wangda Tan commented on YARN-1011:
----------------------------------

Thanks [~kasha] and also comments from [~elgoiri]. Looked at doc, it looks good.

Some questions/comments:
- For resource oversubscription enable/disable for individual nodes, I think it's very important
since some nodes could be more important than others. Do you think is it fine to add a configuration
item to each NM's yarn-site.xml?
- For scheduler-side implementation, instead of modifying individual scheduler, I think we
should try to add over-subscription policy to common scheduling layer since it doesn't sounds
very related to specific scheduler implementation.

I also agree for the first implementation, we can simply assume nodes have more resource to
use. CS shouldn't have issue with this assumption.

> [Umbrella] Schedule containers based on utilization of currently allocated containers
> -------------------------------------------------------------------------------------
>
>                 Key: YARN-1011
>                 URL: https://issues.apache.org/jira/browse/YARN-1011
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Arun C Murthy
>         Attachments: yarn-1011-design-v0.pdf
>
>
> Currently RM allocates containers and assumes resources allocated are utilized.
> RM can, and should, get to a point where it measures utilization of allocated containers
and, if appropriate, allocate more (speculative?) containers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message