hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy Ryza (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1197) Support changing resources of an allocated container
Date Thu, 12 Dec 2013 01:35:08 GMT

    [ https://issues.apache.org/jira/browse/YARN-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13845982#comment-13845982

Sandy Ryza commented on YARN-1197:

bq. Seems like the control flow is asymetrical for resource decrease. We directly go to the
node first. Is that intended? On first look, that seems fine - decreasing resource usage on
a node is akin to killing a container by talking to NM directly.
This is intentional - we went through a few different flows before settling on this approach.
 The analogy with killing the container was one of the reasons for this.

bq. In such applications that decrease container-resource, will the application first instruct
its container to reduce the resource usage and then inform the platform? The reason this is
important is if it doesn't happen that way, node will forcefully either kill it when monitoring
resource usage or change its cgroup immediately causing the container to swap.
When reducing memory, the application should inform the container process before informing
the NodeManager.  When only reducing CPU, there will probably be situations where only informing
the platform is necessary.

bq. To avoid branch-rot, we could target a subset, say just the resource-increase changes
in the branch and do the remaining work on trunk after merge.
Sounds reasonable to me.

> Support changing resources of an allocated container
> ----------------------------------------------------
>                 Key: YARN-1197
>                 URL: https://issues.apache.org/jira/browse/YARN-1197
>             Project: Hadoop YARN
>          Issue Type: Task
>          Components: api, nodemanager, resourcemanager
>    Affects Versions: 2.1.0-beta
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: mapreduce-project.patch.ver.1, tools-project.patch.ver.1, yarn-1197-v2.pdf,
yarn-1197-v3.pdf, yarn-1197-v4.pdf, yarn-1197-v5.pdf, yarn-1197.pdf, yarn-api-protocol.patch.ver.1,
yarn-pb-impl.patch.ver.1, yarn-server-common.patch.ver.1, yarn-server-nodemanager.patch.ver.1,
> The current YARN resource management logic assumes resource allocated to a container
is fixed during the lifetime of it. When users want to change a resource 
> of an allocated container the only way is releasing it and allocating a new container
with expected size.
> Allowing run-time changing resources of an allocated container will give us better control
of resource usage in application side

This message was sent by Atlassian JIRA

View raw message