hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4712) CPU Usage Metric is not captured properly in YARN-2928
Date Fri, 26 Feb 2016 06:20:18 GMT

    [ https://issues.apache.org/jira/browse/YARN-4712?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15168524#comment-15168524

Varun Saxena commented on YARN-4712:

We had decided in YARN-4053 that we wont support doubles/floats as of now because of the complexities
involved due to its implementation.
Moreover, we found that most of the times floats would come for some metric which is represented
as a percentage(%) or a ratio.
In this case the denominator is fixed so you can easily normalize the value and store it as
an integer. And divide it by 100(if its a %) when you read it back.
Moreover, in case of percentages a precision higher than 2 decimal places, wont be mighty

We thought if user/client wanted to store a value/ratio where denominator is not fixed, he
can merely store numerator and denominator separately and do the necessary computation after
reading it back.

We had decided until and unless there was a strong use case, we will go with only longs for
And we could not think of any use cases for current metrics as of now.
And there is a way around it as well if somebody wants to store a float by normalizing the
value and storing numerator and denominator separately.

> CPU Usage Metric is not captured properly in YARN-2928
> ------------------------------------------------------
>                 Key: YARN-4712
>                 URL: https://issues.apache.org/jira/browse/YARN-4712
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: timelineserver
>            Reporter: Naganarasimha G R
>            Assignee: Naganarasimha G R
>              Labels: yarn-2928-1st-milestone
> There are 2 issues with CPU usage collection 
> * I was able to observe that that many times CPU usage got from {{pTree.getCpuUsagePercent()}}
is ResourceCalculatorProcessTree.UNAVAILABLE(i.e. -1) but ContainersMonitor do the calculation
 i.e. {{cpuUsageTotalCoresPercentage = cpuUsagePercentPerCore /resourceCalculatorPlugin.getNumProcessors()}}
because of which UNAVAILABLE check in {{NMTimelinePublisher.reportContainerResourceUsage}}
is not encountered. so proper checks needs to be handled
> * {{EntityColumnPrefix.METRIC}} uses always LongConverter but ContainerMonitor is publishing
decimal values for the CPU usage.

This message was sent by Atlassian JIRA

View raw message