hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4304) AM max resource configuration per partition to be displayed/updated correctly in UI and in various partition related metrics
Date Wed, 30 Dec 2015 17:25:49 GMT

    [ https://issues.apache.org/jira/browse/YARN-4304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15075232#comment-15075232

Sunil G commented on YARN-4304:

Hi [~leftnoteasy]

Thank you very much for sharing the comments. I have few doubts in same.

bq.Could you update user am-limit (in User.resourceUsage) as well when computing user am-limit
at activateApplications? 
We already doing this in activateApplications as below.
user.getResourceUsage().setAMLimit(partitionName, userAMLimit);

Now if we need to do this in {{getUserAMResourceLimitPerPartition}}, we need to pass the username
as well. This user name comes from {{application}}
// Check user am resource limit
      User user = getUser(application.getUser());

1. We cannot pre-compute user am-limit like we have done for am-limit before pendingOrderingPolicy
loop in {{activateApplications}}
2. So as we compute this limit every time, we store in {{user.getResourceUsage()}}. I thought
of resusing this. However there can be cases where 0 user/1 users/Multiple users for one queue.
So getting correct user is not really predictable for all getters (now we do not supply any
user name in getAMUserLimit/Partition, even though we take first user, there can be cases
where 0 users). 


bq. CapacitySchedulerPage, Instead of getAMResourceLimit() , Shouldn't you use getAMResourceLimit(partition)?
      PartitionResourcesInfo resourceUsages =

      // Get UserInfo from first user to calculate AM Resource Limit per user.
      ResourceInfo userAMResourceLimit = null;
      ArrayList<UserInfo> usersList = lqinfo.getUsers().getUsersList();
      if (usersList.isEmpty()) {
        // If no users are present, consider AM Limit for that queue.
        userAMResourceLimit = resourceUsages.getAMResourceLimit();

Here {{resourceUsages}} is already taken for specific label. Hence i think we do not need
per label am-limit. 

> AM max resource configuration per partition to be displayed/updated correctly in UI and
in various partition related metrics
> ----------------------------------------------------------------------------------------------------------------------------
>                 Key: YARN-4304
>                 URL: https://issues.apache.org/jira/browse/YARN-4304
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: webapp
>    Affects Versions: 2.7.1
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: 0001-YARN-4304.patch, 0002-YARN-4304.patch, 0003-YARN-4304.patch,
0004-YARN-4304.patch, 0005-YARN-4304.patch, 0005-YARN-4304.patch, 0006-YARN-4304.patch, 0007-YARN-4304.patch,
> As we are supporting per-partition level max AM resource percentage configuration, UI
and various metrics also need to display correct configurations related to same. 
> For eg: Current UI still shows am-resource percentage per queue level. This is to be
updated correctly when label config is used.
> - Display max-am-percentage per-partition in Scheduler UI (label also) and in ClusterMetrics
> - Update queue/partition related metrics w.r.t per-partition am-resource-percentage

This message was sent by Atlassian JIRA

View raw message