Return-Path: X-Original-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C515918512 for ; Sat, 7 Nov 2015 00:54:11 +0000 (UTC) Received: (qmail 92037 invoked by uid 500); 7 Nov 2015 00:54:11 -0000 Delivered-To: apmail-hadoop-yarn-issues-archive@hadoop.apache.org Received: (qmail 91957 invoked by uid 500); 7 Nov 2015 00:54:11 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: yarn-issues@hadoop.apache.org Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 91920 invoked by uid 99); 7 Nov 2015 00:54:11 -0000 Received: from arcas.apache.org (HELO arcas) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 07 Nov 2015 00:54:11 +0000 Received: from arcas.apache.org (localhost [127.0.0.1]) by arcas (Postfix) with ESMTP id 340BB2C1F72 for ; Sat, 7 Nov 2015 00:54:11 +0000 (UTC) Date: Sat, 7 Nov 2015 00:54:11 +0000 (UTC) From: "Wangda Tan (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (YARN-3980) Plumb resource-utilization info in node heartbeat through to the scheduler MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-3980?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14994799#comment-14994799 ] Wangda Tan commented on YARN-3980: ---------------------------------- Just noticed this ticket. [~sunilg], since this patch is almost ready to go, if you think it's fine, could you update YARN-4292 to base on this one? And could you take a look at this patch to see if there's any changes need to merge? [~goiri], thanks for working on this, two comments {code} 352 public void setContainersUtilization( 353 ResourceUtilization containersUtilization) { 354 if (containersUtilization != null) { 355 this.containersUtilization = containersUtilization; 356 } 357 } {code} It seems SchedulerNode cannot update containerUtilization since it's always not null. I think you should directly update utilization when set. And I suggest to update SchedulerNode.containersUtilization/nodeUtilization to use violate. And for naming, I suggest to change getContainersUtilization in RMNode/SchedulerNode/RMNodeStatusEvent to get*Aggregated*ContainersUtilization, which is more straight forward to me. > Plumb resource-utilization info in node heartbeat through to the scheduler > -------------------------------------------------------------------------- > > Key: YARN-3980 > URL: https://issues.apache.org/jira/browse/YARN-3980 > Project: Hadoop YARN > Issue Type: Bug > Components: resourcemanager, scheduler > Affects Versions: 2.7.1 > Reporter: Karthik Kambatla > Assignee: Inigo Goiri > Attachments: YARN-3980-v0.patch, YARN-3980-v1.patch, YARN-3980-v2.patch, YARN-3980-v3.patch, YARN-3980-v4.patch > > > YARN-1012 and YARN-3534 collect resource utilization information for all containers and the node respectively and send it to the RM on node heartbeat. We should plumb it through to the scheduler so the scheduler can make use of it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)