hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "MENG DING (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1197) Support changing resources of an allocated container
Date Tue, 16 Jun 2015 23:00:12 GMT

    [ https://issues.apache.org/jira/browse/YARN-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14588963#comment-14588963

MENG DING commented on YARN-1197:

[~leftnoteasy], I am certainly OK doing (a). My original frustration was mainly about inconsistency
in RM when doing decrease through NM, now that we have all agreed that decrease should go
through RM, the problem is gone.

So here is the latest proposal:

* Container resource decrease:
AM -> RM -> NM
* Container resource increase:
AM -> RM -> AM(token) -> NM. AM needs to poll status of container before using the
additional allocation.
Of course we need to properly handle token expiration (i.e., NM -> RM communication is
needed to unregister the container from the expirer).

In addition, I do *not* see a need for any response to be set in the {{AllocateResponseProto}}:
* For resource decrease, we can assume it is always successful. 
* For resource increase, we are now doing polling to see if the increase is successful.

Let me know if this makes sense.

> Support changing resources of an allocated container
> ----------------------------------------------------
>                 Key: YARN-1197
>                 URL: https://issues.apache.org/jira/browse/YARN-1197
>             Project: Hadoop YARN
>          Issue Type: Task
>          Components: api, nodemanager, resourcemanager
>    Affects Versions: 2.1.0-beta
>            Reporter: Wangda Tan
>         Attachments: YARN-1197 old-design-docs-patches-for-reference.zip, YARN-1197_Design.pdf
> The current YARN resource management logic assumes resource allocated to a container
is fixed during the lifetime of it. When users want to change a resource 
> of an allocated container the only way is releasing it and allocating a new container
with expected size.
> Allowing run-time changing resources of an allocated container will give us better control
of resource usage in application side

This message was sent by Atlassian JIRA

View raw message