hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4865) Track Reserved resources in ResourceUsage and QueueCapacities
Date Thu, 24 Mar 2016 20:01:25 GMT

    [ https://issues.apache.org/jira/browse/YARN-4865?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15210869#comment-15210869

Wangda Tan commented on YARN-4865:

1. CSQueue: getReservedCapacity is not used by anyone, could be removed.

2. FiCaSchedulerApp:
Shouldn't call
this.queue.decReservedResource(node.getPartition(), resource);
Inside FiCaSchedulerApp, because SchedulerApplicationAttempt shouldn't modify queue's state
Instead you can do at LeafQueue#completedContainer:
        // Book-keeping
        if (removed) {
          // Inform the ordering policy
          orderingPolicy.containerReleased(application, rmContainer);
          releaseResource(clusterResource, application, container.getResource(),
              node.getPartition(), rmContainer, false);

Similarily, instead of call {{queue.incReservedResource}} in SchedulerApplicationAttempt,
we should do this in LeafQueue

3. We need to add sufficient tests for this, since reserved resource will be used by many
downstream modules like UI/REST-API and scheduler internally to make decisions, following
candidates to look at:
- Simple container reserve / simple unreserve / reserved-and-container-completed (like preempted)
/ reserved-and-application-completed (Could be added to TestContainerAllocation)
- Also it will be helpful to add simple tests for container reserve / unreserve for node partitions.
(Could be added to TestContainerAllocationForNodeLabel)
- Some tests could be merged or not necessary needed.

> Track Reserved resources in ResourceUsage and QueueCapacities 
> --------------------------------------------------------------
>                 Key: YARN-4865
>                 URL: https://issues.apache.org/jira/browse/YARN-4865
>             Project: Hadoop YARN
>          Issue Type: Bug
>          Components: resourcemanager
>    Affects Versions: 2.7.2
>            Reporter: Sunil G
>            Assignee: Sunil G
>         Attachments: 0001-YARN-4865.patch
> As discussed in YARN-4678, capture reserved capacity separately in QueueCapcities for
better tracking. 

This message was sent by Atlassian JIRA

View raw message