Return-Path: X-Original-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 9A627180D7 for ; Fri, 11 Dec 2015 06:15:11 +0000 (UTC) Received: (qmail 12388 invoked by uid 500); 11 Dec 2015 06:15:11 -0000 Delivered-To: apmail-hadoop-yarn-issues-archive@hadoop.apache.org Received: (qmail 12338 invoked by uid 500); 11 Dec 2015 06:15:11 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: yarn-issues@hadoop.apache.org Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 12323 invoked by uid 99); 11 Dec 2015 06:15:11 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 11 Dec 2015 06:15:11 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id F3F1D2C1F68 for ; Fri, 11 Dec 2015 06:15:10 +0000 (UTC) Date: Fri, 11 Dec 2015 06:15:10 +0000 (UTC) From: "Akira AJISAKA (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (YARN-3432) Cluster metrics have wrong Total Memory when there is reserved memory on CS MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-3432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15052251#comment-15052251 ] Akira AJISAKA commented on YARN-3432: ------------------------------------- bq. Rethinking this, it's better to change how to calculate availableMB in CapacityScheduler. availableMB should include reservedMB for consistency. or to change how to calculate availableMB in FairScheduler to exclude reservedMB. As [~skaterxu] commented, we need to take care of vcores as well. > Cluster metrics have wrong Total Memory when there is reserved memory on CS > --------------------------------------------------------------------------- > > Key: YARN-3432 > URL: https://issues.apache.org/jira/browse/YARN-3432 > Project: Hadoop YARN > Issue Type: Bug > Components: capacityscheduler, resourcemanager > Affects Versions: 2.6.0 > Reporter: Thomas Graves > Assignee: Brahma Reddy Battula > Attachments: YARN-3432-002.patch, YARN-3432.patch > > > I noticed that when reservations happen when using the Capacity Scheduler, the UI and web services report the wrong total memory. > For example. I have a 300GB of total memory in my cluster. I allocate 50 and I reserve 10. The cluster metrics for total memory get reported as 290GB. > This was broken by https://issues.apache.org/jira/browse/YARN-656 so perhaps there is a difference between fair scheduler and capacity scheduler. -- This message was sent by Atlassian JIRA (v6.3.4#6332)