hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Íñigo Goiri (JIRA) <j...@apache.org>
Subject [jira] [Commented] (YARN-8911) ContainerScheduler incorrectly uses percentage number as the cpu resource utlization
Date Mon, 22 Oct 2018 23:55:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16659851#comment-16659851

Íñigo Goiri commented on YARN-8911:

I don't think we have very good coverage of this code.
Is there any test that would track this from the container to the RM?

> ContainerScheduler incorrectly uses percentage number as the cpu resource utlization
> ------------------------------------------------------------------------------------
>                 Key: YARN-8911
>                 URL: https://issues.apache.org/jira/browse/YARN-8911
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>            Reporter: Haibo Chen
>            Assignee: Haibo Chen
>            Priority: Major
>         Attachments: YARN-8911.00.patch, YARN-8911.01.patch
> *UPDATE*:  *per discussion below, the cpu resource utlization (ResourceUtilzation.cpu)
is incorrectly documented as a percentage number in [0, 1.0] in the java doc. It is however,
used as the number of vcores used throughout the system.  YARN-4597 is a victim of the 
misleading java doc.*
> ResourceUtilization represents the cpu utilization with a float number in [0, 1.0], i.e.
the percentage of cpu usage across the node.  However, when Containers Monitor tracks the
total aggregate resource utilization of all containers, it adds up the total number of vcores
used by all running containers.
> (If you have 6 containers running, each consuming 1 vcore, you'd see the aggregated cpu
container utilization being 6.0, but it's supposed to be always between 0 and 1.0)   See
[the code|https://github.com/apache/hadoop/blob/beb850d8f7f1fefa7a6d9502df2b4a4eea372523/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java#L672]

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message