hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Inigo Goiri (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1011) [Umbrella] Schedule containers based on utilization of currently allocated containers
Date Wed, 13 Jan 2016 21:20:40 GMT

    [ https://issues.apache.org/jira/browse/YARN-1011?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15097009#comment-15097009

Inigo Goiri commented on YARN-1011:

[~kasha] can you add some examples to the doc for clarification?

For example, if a node has 10 CPUs, yarn.nodemanager.overallocation.allocation-threshold is
OT=0.5 and yarn.nodemanager.overallocation.preemption-threshold is PT=1.0. In the regular
case we have 2 containers with an allocation of 5 CPUs each and each uses only 2.5 CPUs. With
this proposal, we could add a third container with an allocation of 5 CPUs which also uses
2.5s CPUs. When would we preempt? How would this evolved when the utilization changes? How
would it promote?

> [Umbrella] Schedule containers based on utilization of currently allocated containers
> -------------------------------------------------------------------------------------
>                 Key: YARN-1011
>                 URL: https://issues.apache.org/jira/browse/YARN-1011
>             Project: Hadoop YARN
>          Issue Type: New Feature
>            Reporter: Arun C Murthy
>         Attachments: yarn-1011-design-v0.pdf, yarn-1011-design-v1.pdf, yarn-1011-design-v2.pdf
> Currently RM allocates containers and assumes resources allocated are utilized.
> RM can, and should, get to a point where it measures utilization of allocated containers
and, if appropriate, allocate more (speculative?) containers.

This message was sent by Atlassian JIRA

View raw message