hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Manikandan R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5179) Issue of CPU usage of containers
Date Sat, 18 Feb 2017 19:29:44 GMT

    [ https://issues.apache.org/jira/browse/YARN-5179?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15873308#comment-15873308
] 

Manikandan R commented on YARN-5179:
------------------------------------

[~asuresh]

Based on my understanding, resourceCalculatorPlugin.getNumProcessors() and maxVCoresAllottedForContainers
difference cause this millivcores calculation issue. resourceCalculatorPlugin.getNumProcessors()
simply return no. of logical processors count, whereas maxVCoresAllottedForContainers has
values based on yarn.nodemanager.resource.count-logical-processors-as-cores property. If above
property is true, then maxVCoresAllottedForContainers is equal to resourceCalculatorPlugin.getNumProcessors(),
otherwise, it differs. I am assuming yarn.nodemanager.resource.detect-hardware-capabilities
has been enabled in this case.

Can we use actual vcores being used?

Instead of 

{code}float cpuUsageTotalCoresPercentage = cpuUsagePercentPerCore / resourceCalculatorPlugin.getNumProcessors();
int milliVcoresUsed = (int) (cpuUsageTotalCoresPercentage * 1000 * maxVCoresAllottedForContainers
/nodeCpuPercentageForYARN);{code}

Can we use this?

{code}float cpuUsageTotalCoresPercentage = cpuUsagePercentPerCore / resourceCalculatorPlugin.getNumVcoresUsed();
int milliVcoresUsed = (int) (cpuUsageTotalCoresPercentage * 1000 * resourceCalculatorPlugin.getNumVcoresUsed()/nodeCpuPercentageForYARN);{code}

Please provide your thoughts.

> Issue of CPU usage of containers
> --------------------------------
>
>                 Key: YARN-5179
>                 URL: https://issues.apache.org/jira/browse/YARN-5179
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>    Affects Versions: 2.7.0
>         Environment: Both on Windows and Linux
>            Reporter: Zhongkai Mi
>
> // Multiply by 1000 to avoid losing data when converting to int 
>        int milliVcoresUsed = (int) (cpuUsageTotalCoresPercentage * 1000 
>               * maxVCoresAllottedForContainers /nodeCpuPercentageForYARN); 
> This formula will not get right CPU usage based vcore if vcores != physical cores. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message