hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1197) Support changing resources of an allocated container
Date Thu, 12 Dec 2013 01:45:08 GMT

    [ https://issues.apache.org/jira/browse/YARN-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13845988#comment-13845988

Wangda Tan commented on YARN-1197:

Hi Vinod,
Thanks for jumping in, my idea on your questions,

Seems like the control flow is asymetrical for resource decrease. We directly go to the node
first. Is that intended? On first look, that seems fine - decreasing resource usage on a node
is akin to killing a container by talking to NM directly.
Yes, we discussed this in this Jira (credit to Bikas, Sandy and Tucu), I think decreasing
resource is a similar operation comparing to kill a container

In such applications that decrease container-resource, will the application first instruct
its container to reduce the resource usage and then inform the platform? The reason this is
important is if it doesn't happen that way, node will forcefully either kill it when monitoring
resource usage or change its cgroup immediately causing the container to swap.
Yes I think, AM should notify NM about this when it make sure resource usage is already reduced
in containers to avoid container killed by NM.

I support to move this to a branch to make it nice completed before merging it to trunk.

> Support changing resources of an allocated container
> ----------------------------------------------------
>                 Key: YARN-1197
>                 URL: https://issues.apache.org/jira/browse/YARN-1197
>             Project: Hadoop YARN
>          Issue Type: Task
>          Components: api, nodemanager, resourcemanager
>    Affects Versions: 2.1.0-beta
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: mapreduce-project.patch.ver.1, tools-project.patch.ver.1, yarn-1197-v2.pdf,
yarn-1197-v3.pdf, yarn-1197-v4.pdf, yarn-1197-v5.pdf, yarn-1197.pdf, yarn-api-protocol.patch.ver.1,
yarn-pb-impl.patch.ver.1, yarn-server-common.patch.ver.1, yarn-server-nodemanager.patch.ver.1,
> The current YARN resource management logic assumes resource allocated to a container
is fixed during the lifetime of it. When users want to change a resource 
> of an allocated container the only way is releasing it and allocating a new container
with expected size.
> Allowing run-time changing resources of an allocated container will give us better control
of resource usage in application side

This message was sent by Atlassian JIRA

View raw message