hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Íñigo Goiri (JIRA) <j...@apache.org>
Subject [jira] [Commented] (YARN-8911) NM incorrectly account for container cpu utilization by their number of vcores
Date Fri, 19 Oct 2018 22:32:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-8911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16657521#comment-16657521
] 

Íñigo Goiri commented on YARN-8911:
-----------------------------------

I think reporting 6 is the correct way to go.
We report as vcores.
The comment in {{ResourceUtilization#getCPU()}} is a little ambiguous but it fits the current
behavior.

In our internal deployment we use this extensively and this is consistent between containers
and nodes.

> NM incorrectly account for container cpu utilization by their number of vcores
> ------------------------------------------------------------------------------
>
>                 Key: YARN-8911
>                 URL: https://issues.apache.org/jira/browse/YARN-8911
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>            Reporter: Haibo Chen
>            Assignee: Haibo Chen
>            Priority: Major
>         Attachments: YARN-8911.00.patch
>
>
> ResourceUtilization represents the cpu utilization with a float number in [0, 1.0], i.e.
the percentage of cpu usage across the node.  However, when Containers Monitor tracks the
total aggregate resource utilization of all containers, it adds up the total number of vcores
used by all running containers.
>  
> (If you have 6 containers running, each consuming 1 vcore, you'd see the aggregated cpu
container utilization being 6.0, but it's supposed to be always between 0 and 1.0)   See
[the code|https://github.com/apache/hadoop/blob/beb850d8f7f1fefa7a6d9502df2b4a4eea372523/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-nodemanager/src/main/java/org/apache/hadoop/yarn/server/nodemanager/containermanager/monitor/ContainersMonitorImpl.java#L672]



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message