hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1197) Support changing resources of an allocated container
Date Tue, 12 May 2015 21:30:02 GMT

    [ https://issues.apache.org/jira/browse/YARN-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14540794#comment-14540794
] 

Wangda Tan commented on YARN-1197:
----------------------------------

[~mding],
Thanks for interesting in this ticket, some comments:
bq. For JVM based containers (e.g., container running HBase), it is not possible right now
to change the heap size of JVM without restarting the Java process. Even if we can implement
a wrapper in the container to relaunch a Java process when resource is changed for a container,
we still need to implement an interface between node manager and container to trigger the
relaunch action.
Good point, this is one thing we noted as well. I don't think there's any easy solution to
shrink JVM. Relaunch the container could be one method, but it will be hard to make a generic
"container wrapper" since kill and relaunch will make data in memory lost.

But since the shrink memory is a proactive action, when a process wants to shrink its resource,
it can use its own "container wrapper" to relaunch the process if it has some data recovery
mechanism.


> Support changing resources of an allocated container
> ----------------------------------------------------
>
>                 Key: YARN-1197
>                 URL: https://issues.apache.org/jira/browse/YARN-1197
>             Project: Hadoop YARN
>          Issue Type: Task
>          Components: api, nodemanager, resourcemanager
>    Affects Versions: 2.1.0-beta
>            Reporter: Wangda Tan
>         Attachments: mapreduce-project.patch.ver.1, tools-project.patch.ver.1, yarn-1197-scheduler-v1.pdf,
yarn-1197-v2.pdf, yarn-1197-v3.pdf, yarn-1197-v4.pdf, yarn-1197-v5.pdf, yarn-1197.pdf, yarn-api-protocol.patch.ver.1,
yarn-pb-impl.patch.ver.1, yarn-server-common.patch.ver.1, yarn-server-nodemanager.patch.ver.1,
yarn-server-resourcemanager.patch.ver.1
>
>
> The current YARN resource management logic assumes resource allocated to a container
is fixed during the lifetime of it. When users want to change a resource 
> of an allocated container the only way is releasing it and allocating a new container
with expected size.
> Allowing run-time changing resources of an allocated container will give us better control
of resource usage in application side



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message