hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1197) Support changing resources of an allocated container
Date Thu, 12 Dec 2013 05:55:12 GMT

    [ https://issues.apache.org/jira/browse/YARN-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13846084#comment-13846084
] 

Wangda Tan commented on YARN-1197:
----------------------------------

My thoughts after reading Bikas and Sandy's comments,
I think the two Jiras are tackling different problem, and like what Sandy said, the container
increase/decrease can give scheduler more inputs to optimize such increase requests (like
user can configure "increase request priority" higher than "normal container request priority",
which can faster handle such increase operation to support low latency services). 
And I think if user want to use container delegation instead of container increasing just
to make more dynamic resource (not shared between users/applications), I've some concerns
beyond Sandy's thinking.

* If we treat all delegated resource as a same container, why not simplely merge them (like
what I originally proposed in this Jira), merge them will make us easier to manage preemption,
etc.
* How to deal with running container merge (a container is already running, and what should
be happen if we delegate its resource to another container). If we can only delegate container
in ACQUIRED state, we need to deal with timeout to launch a container before it taken back
by RM
* Can we do container "de-delegation"?

In general, I support to use container increase/decrease to deal with resource changing within
a single application.

> Support changing resources of an allocated container
> ----------------------------------------------------
>
>                 Key: YARN-1197
>                 URL: https://issues.apache.org/jira/browse/YARN-1197
>             Project: Hadoop YARN
>          Issue Type: Task
>          Components: api, nodemanager, resourcemanager
>    Affects Versions: 2.1.0-beta
>            Reporter: Wangda Tan
>            Assignee: Wangda Tan
>         Attachments: mapreduce-project.patch.ver.1, tools-project.patch.ver.1, yarn-1197-v2.pdf,
yarn-1197-v3.pdf, yarn-1197-v4.pdf, yarn-1197-v5.pdf, yarn-1197.pdf, yarn-api-protocol.patch.ver.1,
yarn-pb-impl.patch.ver.1, yarn-server-common.patch.ver.1, yarn-server-nodemanager.patch.ver.1,
yarn-server-resourcemanager.patch.ver.1
>
>
> The current YARN resource management logic assumes resource allocated to a container
is fixed during the lifetime of it. When users want to change a resource 
> of an allocated container the only way is releasing it and allocating a new container
with expected size.
> Allowing run-time changing resources of an allocated container will give us better control
of resource usage in application side



--
This message was sent by Atlassian JIRA
(v6.1.4#6159)

Mime
View raw message