Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Fri, 29 Jul 2016 09:55:20 +0000 (UTC)
From: "Sunil G (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12993474.1469779412000.167363.1469786120803@Atlassian.JIRA>
In-Reply-To: <JIRA.12993474.1469779412000@Atlassian.JIRA>
References: <JIRA.12993474.1469779412000@Atlassian.JIRA> <JIRA.12993474.1469779412871@arcas>
Subject: [jira] [Commented] (YARN-5448) Resource in Cluster Metrics is not
 sum of resources in all nodes of all partitions
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 29 Jul 2016 09:55:27 -0000


    [ https://issues.apache.org/jira/browse/YARN-5448?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15399069#comment-15399069 ] 

Sunil G commented on YARN-5448:
-------------------------------

From use case perspective, there are 2 use case will pop from user
# I have *N* nodes in my cluster configured with *M* GB of resource. But my cluster resource in web UI is not showing *N x M* resource. (Current behavior)
# I have *X* resource in my cluster and cluster resource of web UI is displaying the same too. But resource allocation is only some % of *X*. (not for full cluster). 
2nd question will pop on later after fixing in the proposed way here.

I think these 2 are debatable topic and we can choose which one we can answer. As from the second part of your proposal, you are planning to show such labels in scheduler UI. But I am not sure how these labels can be visible enough to user from scheduler UI  to convey the issue. Could you share how it may come.

Hence I thought, we can have a column like "Non usable cluster resource" and we can hide if labels are not enabled too. As mentioned we can wait for other folks to pitch in too.

> Resource in Cluster Metrics is not sum of resources in all nodes of all partitions
> ----------------------------------------------------------------------------------
>
>                 Key: YARN-5448
>                 URL: https://issues.apache.org/jira/browse/YARN-5448
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacity scheduler, resourcemanager, webapp
>    Affects Versions: 2.7.2
>            Reporter: Naganarasimha G R
>            Assignee: Naganarasimha G R
>         Attachments: NodesPage.png, schedulerPage.png
>
>
> Currently Resource info from Cluster Metrics are got from Queue Metrics's *available resource + allocated resource*. Hence if there are some nodes which belongs to partition but if that partition is not associated with any queue then in the capacity scheduler partition hierarchy shows this nodes resources under its partition but Cluster metrics doesn't show. 
> Apart from this in the Nodes page too Metrics overview table is shown. So if we show Resource info from Queue Metrics User will not be able to co relate it. (have attached the images for the same)
> IIUC idea of not showing in the *Metrics overview table* is to highlight that configuration is not proper. This needs to be some how conveyed through  parititon-by-queue-hierarchy chart.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org