hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nathan Roberts (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3432) Cluster metrics have wrong Total Memory when there is reserved memory on CS
Date Thu, 01 Sep 2016 20:38:20 GMT

    [ https://issues.apache.org/jira/browse/YARN-3432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15456537#comment-15456537
] 

Nathan Roberts commented on YARN-3432:
--------------------------------------

Recently ran into this issue again. Just seems wrong that totalMB fluctuates significantly
due to reservedMB moving around. Since we can't reserve beyond the size of the cluster, it
seems fine to have Total MB = Available MB + Allocated MB + Reserved MB for the capacity scheduler.


I guess this could be an incompatible change to anyone who's worked around the problem by
adding reservedMB to totalMB. [~vinodkv], [~kasha@cloudera.com], others have comments on this
aspect?


> Cluster metrics have wrong Total Memory when there is reserved memory on CS
> ---------------------------------------------------------------------------
>
>                 Key: YARN-3432
>                 URL: https://issues.apache.org/jira/browse/YARN-3432
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: capacityscheduler, resourcemanager
>    Affects Versions: 2.6.0
>            Reporter: Thomas Graves
>            Assignee: Brahma Reddy Battula
>         Attachments: YARN-3432-002.patch, YARN-3432-003.patch, YARN-3432.patch
>
>
> I noticed that when reservations happen when using the Capacity Scheduler, the UI and
web services report the wrong total memory.
> For example.  I have a 300GB of total memory in my cluster.  I allocate 50 and I reserve
10.  The cluster metrics for total memory get reported as 290GB.
> This was broken by https://issues.apache.org/jira/browse/YARN-656 so perhaps there is
a difference between fair scheduler and capacity scheduler.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message