hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Weiwei Yang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-7784) Cluster metrics is inaccurate when placement constraint is enabled
Date Fri, 26 Jan 2018 02:11:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-7784?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16340454#comment-16340454

Weiwei Yang commented on YARN-7784:

Hi [~asuresh]

Since YARN-7670, FiCaSchedulerApp#accept starts to accept a boolean \{{checkPending}}. And
when placement-constraints.enable is true, this is a false value which causes \{{appSchedulingInfo.allocate}}
was skipped, then the metrics deduction was also skipped. Causing this inaccurate metrics
problem. What was the intention of this change? As long as container is allocated, we need
to keep metrics updated no matter which approach it takes. Please suggest how to fix this.


> Cluster metrics is inaccurate when placement constraint is enabled
> ------------------------------------------------------------------
>                 Key: YARN-7784
>                 URL: https://issues.apache.org/jira/browse/YARN-7784
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: metrics, RM
>            Reporter: Weiwei Yang
>            Priority: Major
> Reproducing steps
>  # Setup a cluster and sets {{yarn.resourcemanager.placement-constraints.enabled}} to
>  # Submit a DS job with placement constraint, such as {{-placement_spec foo=2,NOTIN,NODE,foo}}
>  # Check cluster metrics from http://<RM_ADDRESS>/cluster/apps
> when job is running, {{Containers Running}}, {{Memory Used}} and {{VCore Used}} were
not updated (except AM), metrics from containers allocated by the PlacementProcessor were
not accumulated to the cluster metrics; however when job is done, the resource were deducted.
Then UI displays like following:
>  * Containers Running: -2
>  * Memory Used: -400
>  * VCores Used: -2
> Looks like {{AppSchedulingInfo#updateMetricsForAllocatedContainer}} was not called when
allocating a container.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message