hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Brahma Reddy Battula (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3432) Cluster metrics have wrong Total Memory when there is reserved memory on CS
Date Sun, 30 Oct 2016 18:24:58 GMT

    [ https://issues.apache.org/jira/browse/YARN-3432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15620360#comment-15620360

Brahma Reddy Battula commented on YARN-3432:

[~curino] thanks for commit..and thanks to all for additional review.

[~jhung] Pushed to branch-2.7.

In the following code, {{containersReserved}} should be {{reservedVirtualCores}},Even Akira
mentioned same.
+    if (rs instanceof CapacityScheduler) {
+      this.totalMB = availableMB + allocatedMB + reservedMB;
+      this.totalVirtualCores = availableVirtualCores + allocatedVirtualCores
+          + containersReserved;

As I am planing to raise separate issue for testcase(little tricky,as there are no way to
get total metrics),I will handle as part of that.

As testcase for this can be can be done only through webService..So I feel, Firstly we need
to parameterize {{TestRMWebServices.java}} ,then we can add.

> Cluster metrics have wrong Total Memory when there is reserved memory on CS
> ---------------------------------------------------------------------------
>                 Key: YARN-3432
>                 URL: https://issues.apache.org/jira/browse/YARN-3432
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacityscheduler, resourcemanager
>    Affects Versions: 2.6.0
>            Reporter: Thomas Graves
>            Assignee: Brahma Reddy Battula
>              Labels: oct16-easy
>             Fix For: 2.8.0, 2.7.4
>         Attachments: YARN-3432-002.patch, YARN-3432-003.patch, YARN-3432.patch
> I noticed that when reservations happen when using the Capacity Scheduler, the UI and
web services report the wrong total memory.
> For example.  I have a 300GB of total memory in my cluster.  I allocate 50 and I reserve
10.  The cluster metrics for total memory get reported as 290GB.
> This was broken by https://issues.apache.org/jira/browse/YARN-656 so perhaps there is
a difference between fair scheduler and capacity scheduler.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message