hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5448) Resource in Cluster Metrics is not sum of resources in all nodes of all partitions
Date Fri, 29 Jul 2016 08:34:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15398962#comment-15398962

Sunil G commented on YARN-5448:

bq.Metrics overview table needs to show the resources of all the active NM'
[~Naganarasimha Garla]. I think its better to separate non-configured label resources as separate
column in metrics table. Few advantages
- Admin can immediately notice that there are some label which are not-configured or part
of cluster resource , hence cannot be used. Any action can be taken with one look in metrics
table. As I see, admins may choose not to configure any label due reasons like a) some labels
need to be taken out of rotation b) may be a configuration mistake etc. 
- As per proposed approach, if we add up non-configured label resources to cluster resources,
we need to go to scheduler page to get details. Yes, its a perception of seeing the data.
I think its better if metrics are available separately in the very first place in cluster
metrics table. 
- I think we can also have the approach in scheduler to have some info message for non-used
or configured labels. It will definitely help to go for indepth analysis.


> Resource in Cluster Metrics is not sum of resources in all nodes of all partitions
> ----------------------------------------------------------------------------------
>                 Key: YARN-5448
>                 URL: https://issues.apache.org/jira/browse/YARN-5448
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacity scheduler, resourcemanager, webapp
>    Affects Versions: 2.7.2
>            Reporter: Naganarasimha G R
>            Assignee: Naganarasimha G R
>         Attachments: NodesPage.png, schedulerPage.png
> Currently Resource info from Cluster Metrics are got from Queue Metrics's *available
resource + allocated resource*. Hence if there are some nodes which belongs to partition but
if that partition is not associated with any queue then in the capacity scheduler partition
hierarchy shows this nodes resources under its partition but Cluster metrics doesn't show.

> Apart from this in the Nodes page too Metrics overview table is shown. So if we show
Resource info from Queue Metrics User will not be able to co relate it. (have attached the
images for the same)
> IIUC idea of not showing in the *Metrics overview table* is to highlight that configuration
is not proper. This needs to be some how conveyed through  parititon-by-queue-hierarchy chart.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message