hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Íñigo Goiri (JIRA) <j...@apache.org>
Subject [jira] [Commented] (YARN-8911) ContainerScheduler incorrectly uses percentage number as the cpu resource utlization
Date Tue, 23 Oct 2018 18:24:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16661096#comment-16661096

Íñigo Goiri commented on YARN-8911:

Shouldn't the container scheduler test then be affected by this change?
If not, we should add some test to make sure the value is something >1.

> ContainerScheduler incorrectly uses percentage number as the cpu resource utlization
> ------------------------------------------------------------------------------------
>                 Key: YARN-8911
>                 URL: https://issues.apache.org/jira/browse/YARN-8911
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>            Reporter: Haibo Chen
>            Assignee: Haibo Chen
>            Priority: Major
>         Attachments: YARN-8911.00.patch, YARN-8911.01.patch
> *UPDATE*:  *per discussion below, the cpu resource utlization (ResourceUtilzation.cpu)
is incorrectly documented as a percentage number in [0, 1.0] in the java doc. It is however,
used as the number of vcores used throughout the system.  YARN-4597 is a victim of the 
misleading java doc.*
> ResourceUtilization represents the cpu utilization with a float number in [0, 1.0], i.e.
the percentage of cpu usage across the node.  However, when Containers Monitor tracks the
total aggregate resource utilization of all containers, it adds up the total number of vcores
used by all running containers.
> (If you have 6 containers running, each consuming 1 vcore, you'd see the aggregated cpu
container utilization being 6.0, but it's supposed to be always between 0 and 1.0)   See
[the code|https://github.com/apache/hadoop/blob/beb850d8f7f1fefa7a6d9502df2b4a4eea372523/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java#L672]

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message