hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4304) AM max resource configuration per partition to be displayed/updated correctly in UI and in various partition related metrics
Date Mon, 21 Dec 2015 19:14:46 GMT

    [ https://issues.apache.org/jira/browse/YARN-4304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15066912#comment-15066912

Wangda Tan commented on YARN-4304:

Thanks for update [~sunilg],

Some minor comments:
1) CapacitySchedulerPage:
- You can fetch {{lqinfo.getUsers().getUsersList()}} only once.
- {{(resourceUsages.getAmUsed() == null) ? "N/A"}}, is it better to use Resource.None() instead
of N/A?

2) LeafQueue:
- I'm not sure if this is required:
  public synchronized Resource getAMResourceLimit() {
    // Ensure we calculate limit when its not pre-computed
    if (queueUsage.getAMLimit().equals(Resources.none())) {
Since calculateAndGetAMResourceLimit is called by activateApplications, and activateApplications
is called by updateClusterResource. It will be updated when cluster resource changed or queue
configuration changed (initialized).
I think the getAMResourceLimit should safely return queueUsage.getAMLimit directly.
- getAMResourceLimit doesn't need synchronized lock 
- getUserAMResourceLimit is used by Tests and REST API only. I think REST API can use Resource
from UsersInfo and AMResourceLimit, no need to access queue's synchronized lock. And I think
you can move following code to CapacitySchedulerLeafQueueInfo:
140	      // Get UserInfo from first user to calculate AM Resource Limit per user.
141	      ResourceInfo userAMResourceLimit = null;
142	      if (lqinfo.getUsers().getUsersList().isEmpty()) {
143	        // If no users are present, consider AM Limit for that queue.
144	        userAMResourceLimit = resourceUsages.getAMResourceLimit();
145	      } else {
146	        userAMResourceLimit = lqinfo.getUsers().getUsersList().get(0)
147	            .getResourceUsageInfo().getPartitionResourceUsageInfo(label)
148	            .getAMResourceLimit();
149	      }

> AM max resource configuration per partition to be displayed/updated correctly in UI and
in various partition related metrics
> ----------------------------------------------------------------------------------------------------------------------------
>                 Key: YARN-4304
>                 URL: https://issues.apache.org/jira/browse/YARN-4304
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: webapp
>    Affects Versions: 2.7.1
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: 0001-YARN-4304.patch, 0002-YARN-4304.patch, 0003-YARN-4304.patch,
0004-YARN-4304.patch, 0005-YARN-4304.patch, 0005-YARN-4304.patch, 0006-YARN-4304.patch, REST_and_UI.zip
> As we are supporting per-partition level max AM resource percentage configuration, UI
and various metrics also need to display correct configurations related to same. 
> For eg: Current UI still shows am-resource percentage per queue level. This is to be
updated correctly when label config is used.
> - Display max-am-percentage per-partition in Scheduler UI (label also) and in ClusterMetrics
> - Update queue/partition related metrics w.r.t per-partition am-resource-percentage

This message was sent by Atlassian JIRA

View raw message