hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6870) ResourceUtilization/ContainersMonitorImpl is calculating CPU utilization as a float, which is imprecise
Date Wed, 26 Jul 2017 22:20:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16102371#comment-16102371
] 

Arun Suresh commented on YARN-6870:
-----------------------------------

@Brook Zhou, I think we should start with addressing the bug in this jira. We can probably
change in the internal protobuf representation if we find it useful for other downstream features
like YARN-1011.


> ResourceUtilization/ContainersMonitorImpl is calculating CPU utilization as a float,
which is imprecise
> -------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-6870
>                 URL: https://issues.apache.org/jira/browse/YARN-6870
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: api, nodemanager
>            Reporter: Brook Zhou
>            Assignee: Brook Zhou
>
> We have seen issues on our clusters where the current way of computing CPU usage is having
float-arithmetic inaccuracies (the bug is still there in trunk)
> Simple program to illustrate:
> {code:title=Bar.java|borderStyle=solid}
>       public static void main(String[] args) throws Exception {
>             float result = 0.0f;
>             for (int i = 0; i < 7; i++) {
>                   if (i == 6) {
>                         result += (float) 4 / (float)18;
>                   } else {
>                         result += (float) 2 / (float)18;
>                   }
>             }
>             for (int i = 0; i < 7; i++) {
>                   if (i == 6) {
>                         result -= (float) 4 / (float)18;
>                   } else {
>                         result -= (float) 2 / (float)18;
>                   }                 
>             }
>             System.out.println(result);
>       }
> {code}
> // Printed
> 4.4703484E-8
> 2017-04-12 05:43:24,014 INFO org.apache.hadoop.yarn.server.nodemanager.containermanager.monitor.ContainersMonitorImpl:
Not enough cpu for [container_e3295_1491978508342_0467_01_000030], Current CPU Allocation:
[0.8888891], Requested CPU Allocation: [0.11111111]
> There are a few places with this issue:
> 1. ResourceUtilization.java - set/getCPU both use float. When ContainerScheduler calls
ContainersMonitor.increase/decreaseResourceUtilization, this may lead to issues.
> 2. AllocationBasedResourceUtilizationTracker.java  - hasResourcesAvailable uses float
as well for CPU computation.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message