hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Varun Saxena (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (YARN-5553) [ATSv2] "metrics" for YARN_CONTAINERS are displayed as empty.
Date Tue, 23 Aug 2016 16:53:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-5553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15433181#comment-15433181
] 

Varun Saxena edited comment on YARN-5553 at 8/23/16 4:52 PM:
-------------------------------------------------------------

[~rohithsharma], thanks for filing the issue.

bq. The CPU-Memory values taken are from current usage which monitor thread reports. But issue
with it is aggregationOp is SUM. So Each time monitor thread reported values is being added
and give some bigger values. But ideally it is not actual usage.
Yes, we want to aggregate usage which is being reported at app level. This is intentional.
We also plan to do accumulation of metrics along time dimension (effectively area under the
curve). Also follow the discussion on YARN-4712.

bq. Some containers has metric values CPU-Memory, some are empty.
The container usage will be reported based on configuration of monitoring interval. Unlikely
that this would be missed but also, please check if you have enabled timeline service on each
of the NMs' where containers were allocated.

bq. There can be seen that master containers has metrics and also observe that memory value
is very huge
Memory reported is in bytes.


was (Author: varun_saxena):
[~rohithsharma], thanks for filing the issue.

bq. The CPU-Memory values taken are from current usage which monitor thread reports. But issue
with it is aggregationOp is SUM. So Each time monitor thread reported values is being added
and give some bigger values. But ideally it is not actual usage.
Yes, we want to aggregated usage which is being reported at app level. This is intentional.
We also plan to do accumulation of metrics along time dimension (effectively area under the
curve). Also follow the discussion on YARN-4712.

bq. Some containers has metric values CPU-Memory, some are empty.
The container usage will be reported based on configuration of monitoring interval. Unlikely
that this would be missed but also, please check if you have enabled timeline service on each
of the NMs' where containers were allocated.

bq. There can be seen that master containers has metrics and also observe that memory value
is very huge
Memory reported is in bytes.

> [ATSv2] "metrics" for YARN_CONTAINERS are displayed as empty.
> -------------------------------------------------------------
>
>                 Key: YARN-5553
>                 URL: https://issues.apache.org/jira/browse/YARN-5553
>             Project: Hadoop YARN
>          Issue Type: Bug
>            Reporter: Rohith Sharma K S
>         Attachments: rest-output.txt
>
>
> metrics for YARN_CONTAINER entity types are displayed as empty. For master containers
some values are displayed but for other containers randomly it displays metric values.
> I think the behavior is due to publishing of container metric in monitor thread from
NodeManager. There are 2 issues I observe is 
> # Some containers has metric values CPU-Memory, some are empty. 
> # The CPU-Memory values taken are from current usage which monitor thread reports. But
issue with it is *aggregationOp* is SUM. So Each time monitor thread reported values is being
added and give some bigger values. But ideally it is not actual usage. 
> I think CPU-MEMORY metric values for yarn-containers entities should be allocated values.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message