hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Karthik Kambatla (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3304) Default return values for resources in ResourceCalculatorProcessTree are inconsistent in case where functionality is not available.
Date Mon, 09 Mar 2015 04:54:38 GMT

    [ https://issues.apache.org/jira/browse/YARN-3304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14352549#comment-14352549
] 

Karthik Kambatla commented on YARN-3304:
----------------------------------------

Sample output when logging this metric. Note that, by default, these metrics are published
when a container completes. It is a little odd for the CPU usage to be negative. Also, it
is unavailable primarily when the container has not started yet or enough time hasn't elapsed
since start; showing zero for these cases seems reasonable. What do others think? 

{noformat}
kkambatl@pdsl24:~$ tail -f /tmp/nm-container-usage.out
1425872341126 container.ContainerResource_container_1425871888831_0001_01_000013: ContainerResource=container_1425871888831_0001_01_000013,
Context=container, ContainerPid=8874, Hostname=pdsl24.cs.purdue.edu, PMemUsageMBsNumUsage=299,
PMemUsageMBsAvgMBs=161.25083612040132, PMemUsageMBsStdevMBs=15.650566309518153, PMemUsageMBsIMinMBs=45.0,
PMemUsageMBsIMaxMBs=171.0, PMemUsageMBsMinMBs=45.0, PMemUsageMBsMaxMBs=171.0, PCpuUsagePercentNumUsage=299,
PCpuUsagePercentAvgPercents=2.0535117056856196, PCpuUsagePercentStdevPercents=16.322375613869188,
PCpuUsagePercentIMinPercents=-1.0, PCpuUsagePercentIMaxPercents=194.0, PCpuUsagePercentMinPercents=-1.0,
PCpuUsagePercentMaxPercents=194.0, MilliVcoreUsageNumUsage=299, MilliVcoreUsageAvgMilliVcores=24.40802675585284,
MilliVcoreUsageStdevMilliVcores=163.1979921241116, MilliVcoreUsageIMinMilliVcores=-10.0, MilliVcoreUsageIMaxMilliVcores=1940.0,
MilliVcoreUsageMinMilliVcores=-10.0, MilliVcoreUsageMaxMilliVcores=1940.0, pMemLimitMBs=1024,
vMemLimitMBs=2150, vCoreLimit=1
1425872341127 container.ContainerResource_container_1425871888831_0001_01_000005: ContainerResource=container_1425871888831_0001_01_000005,
Context=container, ContainerPid=8878, Hostname=pdsl24.cs.purdue.edu, PMemUsageMBsNumUsage=299,
PMemUsageMBsAvgMBs=153.38127090301018, PMemUsageMBsStdevMBs=14.116311560413372, PMemUsageMBsIMinMBs=50.0,
PMemUsageMBsIMaxMBs=162.0, PMemUsageMBsMinMBs=50.0, PMemUsageMBsMaxMBs=162.0, PCpuUsagePercentNumUsage=299,
PCpuUsagePercentAvgPercents=2.070234113712376, PCpuUsagePercentStdevPercents=16.222811543457478,
PCpuUsagePercentIMinPercents=-1.0, PCpuUsagePercentIMaxPercents=178.0, PCpuUsagePercentMinPercents=-1.0,
PCpuUsagePercentMaxPercents=178.0, MilliVcoreUsageNumUsage=299, MilliVcoreUsageAvgMilliVcores=24.53846153846158,
MilliVcoreUsageStdevMilliVcores=162.46689190542918, MilliVcoreUsageIMinMilliVcores=-10.0,
MilliVcoreUsageIMaxMilliVcores=1785.0, MilliVcoreUsageMinMilliVcores=-10.0, MilliVcoreUsageMaxMilliVcores=1785.0,
pMemLimitMBs=1024, vMemLimitMBs=2150, vCoreLimit=1
{noformat}

> Default return values for resources in ResourceCalculatorProcessTree are inconsistent
in case where functionality is not available.
> -----------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: YARN-3304
>                 URL: https://issues.apache.org/jira/browse/YARN-3304
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: nodemanager
>            Reporter: Junping Du
>            Assignee: Karthik Kambatla
>            Priority: Blocker
>
> Per discussions in YARN-3296, getCpuUsagePercent() will return -1 for unavailable case
while other resource metrics are return 0 in the same case which sounds inconsistent.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message