hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3272) Surface container locality info
Date Sat, 28 Feb 2015 01:09:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-3272?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14341139#comment-14341139
] 

Wangda Tan commented on YARN-3272:
----------------------------------

Hi [~jianhe],
This is very useful! Just took a look, some comments:

1) 
LeafQueue.assignNodeLocalContainers and 2 others, can you use MutableObject instead of the
array passed in?

2) 
Logics in LeafQueue.assignContainersOnNode could be simplified, you can save the resource-request.NodeType
by a reference, which can avoid 3 if statements like: 
{code}
	1313	      // update locality statistics
1314	      if (!allocatedContainers.isEmpty()) {
1315	        if (nodeLocalResourceRequest == null
1316	            && rackLocalResourceRequest == null) {
1317	          application.incNumAllocatedContainers(NodeType.OFF_SWITCH,
1318	            NodeType.OFF_SWITCH);
1319	        } else if (nodeLocalResourceRequest == null
1320	            && rackLocalResourceRequest != null) {
1321	          application.incNumAllocatedContainers(NodeType.OFF_SWITCH,
1322	            NodeType.RACK_LOCAL);
1323	        } else if (nodeLocalResourceRequest != null
1324	            && rackLocalResourceRequest != null) {
1325	          application.incNumAllocatedContainers(NodeType.OFF_SWITCH,
1326	            NodeType.NODE_LOCAL);
1327	        }
1328	        LOG.info("allocatedContainers  " + allocatedContainers.get(0));
1329	
1330	      }
{code}

3) 
This line in AppBlock can be simplified to:
{code}
235	        .td(i == 0 ? "" : (i == 1 ? "" :
236	          String.valueOf(attemptMetrics.getLocalityStatistics()[i][2])))._();
{code}

To be {{.td(i <= 1 ? : "" : ...}}?



> Surface container locality info 
> --------------------------------
>
>                 Key: YARN-3272
>                 URL: https://issues.apache.org/jira/browse/YARN-3272
>             Project: Hadoop YARN
>          Issue Type: Improvement
>            Reporter: Jian He
>            Assignee: Jian He
>         Attachments: YARN-3272.1.patch, container locality table.png
>
>
> We can surface the container locality info on the web UI. This is useful to debug "why
my applications are progressing slow", especially when locality is bad.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message