hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sunil G (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-3216) Max-AM-Resource-Percentage should respect node labels
Date Tue, 06 Oct 2015 02:29:27 GMT

    [ https://issues.apache.org/jira/browse/YARN-3216?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14944416#comment-14944416
] 

Sunil G commented on YARN-3216:
-------------------------------

Thank you [~leftnoteasy] for sharing the comments.

bq.Please let me know if there's any specific reason to add a new maximum-am-resource-partition.
I agree with you. We could use the same configuration name under each label.

bq.if there's any activated apps under a partition, it is equivalent to queueUsage.getAMUsed(partitionName)
Yes. This will be enough, i kept a new map with the idea of maintaining some more information
in similar lines with User. But as of now, the change suggested is enough. I will remove the
map.

bq.you don't have to update am-resource when AM container just allocated, because AM-container.resource
and am-resource-request.node-label-request won't be changed, but you need to update this if
partition of AM-container's NM updated
As I see it, we may need to below change.
- In FiCaSchedulerApp's ctor, update AM-Resource-Request.resource on partition ( keep existing
code). But use {{rmApp.getAMResourceRequest().getNodeLabelExpression()}} to setAMResource
instead of setting to NO_LABEL. Because this information wont be changed later.
- if partition of AM-container's NM updated, we need to change AMResource which I am handling
in {{nodePartitionUpdated}} as below.
{code}
+    if (rmContainer.isAMContainer()) {
+      setAppAMNodePartitionName(newPartition);
+      this.attemptResourceUsage.decAMUsed(oldPartition, containerResource);
+      this.attemptResourceUsage.incAMUsed(newPartition, containerResource);
+      getCSLeafQueue().decAMUsedResource(oldPartition, containerResource, this);
+      getCSLeafQueue().incAMUsedResource(newPartition, containerResource, this);
+    }
{code}
Here AM-Resource-Request.resource is updated in FiCaSchedulerApp's ctor based on {{rmApp.getAMResourceRequest}}.
Once container is allocated, this resource will be come a part of the partition with no change
in resource. So I feel I need not have to update resource in *allocate* call of FicaSchedulerApp.
Am I correct?
- am-resource-percent per user per partition: Yes, I will raise a new ticket to handle this
and will make changes there instead of doing in this.


> Max-AM-Resource-Percentage should respect node labels
> -----------------------------------------------------
>
>                 Key: YARN-3216
>                 URL: https://issues.apache.org/jira/browse/YARN-3216
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Wangda Tan
>            Assignee: Sunil G
>            Priority: Critical
>         Attachments: 0001-YARN-3216.patch, 0002-YARN-3216.patch, 0003-YARN-3216.patch
>
>
> Currently, max-am-resource-percentage considers default_partition only. When a queue
can access multiple partitions, we should be able to compute max-am-resource-percentage based
on that.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message