hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5448) Resource in Cluster Metrics is not sum of resources in all nodes of all partitions
Date Fri, 29 Jul 2016 09:55:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15399069#comment-15399069

Sunil G commented on YARN-5448:

>From use case perspective, there are 2 use case will pop from user
# I have *N* nodes in my cluster configured with *M* GB of resource. But my cluster resource
in web UI is not showing *N x M* resource. (Current behavior)
# I have *X* resource in my cluster and cluster resource of web UI is displaying the same
too. But resource allocation is only some % of *X*. (not for full cluster). 
2nd question will pop on later after fixing in the proposed way here.

I think these 2 are debatable topic and we can choose which one we can answer. As from the
second part of your proposal, you are planning to show such labels in scheduler UI. But I
am not sure how these labels can be visible enough to user from scheduler UI  to convey the
issue. Could you share how it may come.

Hence I thought, we can have a column like "Non usable cluster resource" and we can hide if
labels are not enabled too. As mentioned we can wait for other folks to pitch in too.

> Resource in Cluster Metrics is not sum of resources in all nodes of all partitions
> ----------------------------------------------------------------------------------
>                 Key: YARN-5448
>                 URL: https://issues.apache.org/jira/browse/YARN-5448
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacity scheduler, resourcemanager, webapp
>    Affects Versions: 2.7.2
>            Reporter: Naganarasimha G R
>            Assignee: Naganarasimha G R
>         Attachments: NodesPage.png, schedulerPage.png
> Currently Resource info from Cluster Metrics are got from Queue Metrics's *available
resource + allocated resource*. Hence if there are some nodes which belongs to partition but
if that partition is not associated with any queue then in the capacity scheduler partition
hierarchy shows this nodes resources under its partition but Cluster metrics doesn't show.

> Apart from this in the Nodes page too Metrics overview table is shown. So if we show
Resource info from Queue Metrics User will not be able to co relate it. (have attached the
images for the same)
> IIUC idea of not showing in the *Metrics overview table* is to highlight that configuration
is not proper. This needs to be some how conveyed through  parititon-by-queue-hierarchy chart.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message