hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Naganarasimha G R (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5448) Resource in Cluster Metrics is not sum of resources in all nodes of all partitions
Date Wed, 03 Aug 2016 05:05:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15405320#comment-15405320
] 

Naganarasimha G R commented on YARN-5448:
-----------------------------------------

Thanks for sharing your thoughts [~wangda],

bq. Sorry I may not quite sure about this. Could you explain?
What i meant was, these additional  non-usable resources.columns as part of cluster metrics
table will be use full only when there is a configuration error and once corrected these columns
are not of much use, basically these columns purpose will be almost nill if configured correctly.
One alternative i can think of is show these columns only when partitions are not mapped to
queues. and if value is zero then dont show, thoughts ?

bq.  which can help answering questions like "why I cannot fully utilize the cluster".
One view point what i had for this was captured in the above [comment | https://issues.apache.org/jira/browse/YARN-5448?focusedCommentId=15399248&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15399248],
but well again its a view point and debatable so dont have any hard restrictions on having
it.

bq. It's better to add a non-usable nodes as a separate col, but to me it may not a fully
replacement of total non-usable resources.
May be i did not get the rationale behind {{"total non-usable resources"}} would be better
than {{"non-usable nodes"}}, can elaborate more on your view on this ? 

> Resource in Cluster Metrics is not sum of resources in all nodes of all partitions
> ----------------------------------------------------------------------------------
>
>                 Key: YARN-5448
>                 URL: https://issues.apache.org/jira/browse/YARN-5448
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacity scheduler, resourcemanager, webapp
>    Affects Versions: 2.7.2
>            Reporter: Naganarasimha G R
>            Assignee: Naganarasimha G R
>         Attachments: NodesPage.png, schedulerPage.png
>
>
> Currently Resource info from Cluster Metrics are got from Queue Metrics's *available
resource + allocated resource*. Hence if there are some nodes which belongs to partition but
if that partition is not associated with any queue then in the capacity scheduler partition
hierarchy shows this nodes resources under its partition but Cluster metrics doesn't show.

> Apart from this in the Nodes page too Metrics overview table is shown. So if we show
Resource info from Queue Metrics User will not be able to co relate it. (have attached the
images for the same)
> IIUC idea of not showing in the *Metrics overview table* is to highlight that configuration
is not proper. This needs to be some how conveyed through  parititon-by-queue-hierarchy chart.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message