hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1509) Make AMRMClient support send increase container request and get increased/decreased containers
Date Tue, 29 Sep 2015 18:46:04 GMT

    [ https://issues.apache.org/jira/browse/YARN-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14935618#comment-14935618
] 

Wangda Tan commented on YARN-1509:
----------------------------------

Thanks [~mding],

I think patch generally looks good, one query:
{code}
          // increase/decrease requests could have been added during the
          // allocate call. Those are the newest requests which take precedence
          // over requests cached in the increaseList and decreaseList.
          //
          // Only insert entries from the cached increaseList and decreaseList
          // that do not exist in either of current decrease and increase maps:
          // 1. If the cached increaseList contains the same container as that
          //    in the new increase map, then there is nothing to do as the
          //    the request in the new increase map has the latest value.
          // 2. If the cached increaseList contains the same container as that
          //    in the new decrease map, then there is nothing to do either as
          //    the request in the new decrease map is newer and should cancel
          //    the old increase request.
          // 3. The above also apply to the decreaseList.
          for (ContainerResourceChangeRequest oldIncrease : increaseList) {
            ContainerId oldContainerId = oldIncrease.getContainerId();
            if (increase.get(oldContainerId) == null
                && decrease.get(oldContainerId) == null) {
              increase.put(oldContainerId, oldIncrease.getCapability());
            }
          }
          for (ContainerResourceChangeRequest oldDecrease : decreaseList) {
            ContainerId oldContainerId = oldDecrease.getContainerId();
            if (decrease.get(oldContainerId) == null
                && increase.get(oldContainerId) == null) {
              decrease.put(oldContainerId, oldDecrease.getCapability());
            }
          }
{code}

I think we can simply add decreaseList to decrease and increaseList to increase. If AllocateResponse
== null, we assume allocation fails, and scheduler's increase/decrease table isn't updated.
In this case, I think we should simply revert changes to increase/decrease table. Thoughts?

And I think we can add some debug/info message as well. For example, at {{removePendingChangeRequests}},
if request matches, we can print some logs to show this.

> Make AMRMClient support send increase container request and get increased/decreased containers
> ----------------------------------------------------------------------------------------------
>
>                 Key: YARN-1509
>                 URL: https://issues.apache.org/jira/browse/YARN-1509
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: resourcemanager
>            Reporter: Wangda Tan (No longer used)
>            Assignee: MENG DING
>         Attachments: YARN-1509.1.patch, YARN-1509.2.patch
>
>
> As described in YARN-1197, we need add API in AMRMClient to support
> 1) Add increase request
> 2) Can get successfully increased/decreased containers from RM



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message