Return-Path: X-Original-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Delivered-To: apmail-hadoop-yarn-issues-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 23EEA1755B for ; Sat, 28 Feb 2015 01:09:05 +0000 (UTC) Received: (qmail 34184 invoked by uid 500); 28 Feb 2015 01:09:05 -0000 Delivered-To: apmail-hadoop-yarn-issues-archive@hadoop.apache.org Received: (qmail 34135 invoked by uid 500); 28 Feb 2015 01:09:05 -0000 Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: yarn-issues@hadoop.apache.org Delivered-To: mailing list yarn-issues@hadoop.apache.org Received: (qmail 34124 invoked by uid 99); 28 Feb 2015 01:09:04 -0000 Received: from arcas.apache.org (HELO arcas.apache.org) (140.211.11.28) by apache.org (qpsmtpd/0.29) with ESMTP; Sat, 28 Feb 2015 01:09:04 +0000 Date: Sat, 28 Feb 2015 01:09:04 +0000 (UTC) From: "Wangda Tan (JIRA)" To: yarn-issues@hadoop.apache.org Message-ID: In-Reply-To: References: Subject: [jira] [Commented] (YARN-3272) Surface container locality info MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 7bit X-JIRA-FingerPrint: 30527f35849b9dde25b450d4833f0394 [ https://issues.apache.org/jira/browse/YARN-3272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14341139#comment-14341139 ] Wangda Tan commented on YARN-3272: ---------------------------------- Hi [~jianhe], This is very useful! Just took a look, some comments: 1) LeafQueue.assignNodeLocalContainers and 2 others, can you use MutableObject instead of the array passed in? 2) Logics in LeafQueue.assignContainersOnNode could be simplified, you can save the resource-request.NodeType by a reference, which can avoid 3 if statements like: {code} 1313 // update locality statistics 1314 if (!allocatedContainers.isEmpty()) { 1315 if (nodeLocalResourceRequest == null 1316 && rackLocalResourceRequest == null) { 1317 application.incNumAllocatedContainers(NodeType.OFF_SWITCH, 1318 NodeType.OFF_SWITCH); 1319 } else if (nodeLocalResourceRequest == null 1320 && rackLocalResourceRequest != null) { 1321 application.incNumAllocatedContainers(NodeType.OFF_SWITCH, 1322 NodeType.RACK_LOCAL); 1323 } else if (nodeLocalResourceRequest != null 1324 && rackLocalResourceRequest != null) { 1325 application.incNumAllocatedContainers(NodeType.OFF_SWITCH, 1326 NodeType.NODE_LOCAL); 1327 } 1328 LOG.info("allocatedContainers " + allocatedContainers.get(0)); 1329 1330 } {code} 3) This line in AppBlock can be simplified to: {code} 235 .td(i == 0 ? "" : (i == 1 ? "" : 236 String.valueOf(attemptMetrics.getLocalityStatistics()[i][2])))._(); {code} To be {{.td(i <= 1 ? : "" : ...}}? > Surface container locality info > -------------------------------- > > Key: YARN-3272 > URL: https://issues.apache.org/jira/browse/YARN-3272 > Project: Hadoop YARN > Issue Type: Improvement > Reporter: Jian He > Assignee: Jian He > Attachments: YARN-3272.1.patch, container locality table.png > > > We can surface the container locality info on the web UI. This is useful to debug "why my applications are progressing slow", especially when locality is bad. -- This message was sent by Atlassian JIRA (v6.3.4#6332)