hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "MENG DING (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1197) Support changing resources of an allocated container
Date Wed, 27 May 2015 15:27:23 GMT

    [ https://issues.apache.org/jira/browse/YARN-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14561134#comment-14561134
] 

MENG DING commented on YARN-1197:
---------------------------------

Correct a typo in my previous post, it should be:
bq. As an example, if a container is currently using 2G, and AM asks to increase its resource
to 4G, and then asks again to increase to 6G, but AM doesn't actually use any of the token
to increase the resource on NM. In this case, with the current design, RM can only revert
the resource allocation back to 4G after expiration, not 2G."

Forgot to discuss another important piece. We probably should not use the existing ResourceCalculator
to compare two resource capabilities in this project, because:
- The DefaultResourceCalculator only compares memory, which won't work if we want to only
change CPU cores.
- The DominantResourceCalculator may end up comparing different dimensions between two Resources,
which doesn't make sense in our project.

The way to compare two resource in this project should be straightforward as follows. Let
me know if you think otherwise.
- For increase request, no dimension in the target resource can be smaller than the corresponding
dimension in the current resource, and at least one dimension in the target resource must
be larger than the corresponding dimension in the current resource.
- For decrease request, no dimension in the target resource can be larger than the corresponding
dimension in the current resource, and at least one dimension in the target resource must
be smaller than the corresponding dimension in the current resource. 



> Support changing resources of an allocated container
> ----------------------------------------------------
>
>                 Key: YARN-1197
>                 URL: https://issues.apache.org/jira/browse/YARN-1197
>             Project: Hadoop YARN
>          Issue Type: Task
>          Components: api, nodemanager, resourcemanager
>    Affects Versions: 2.1.0-beta
>            Reporter: Wangda Tan
>         Attachments: YARN-1197_Design.pdf, mapreduce-project.patch.ver.1, tools-project.patch.ver.1,
yarn-1197-scheduler-v1.pdf, yarn-1197-v2.pdf, yarn-1197-v3.pdf, yarn-1197-v4.pdf, yarn-1197-v5.pdf,
yarn-1197.pdf, yarn-api-protocol.patch.ver.1, yarn-pb-impl.patch.ver.1, yarn-server-common.patch.ver.1,
yarn-server-nodemanager.patch.ver.1, yarn-server-resourcemanager.patch.ver.1
>
>
> The current YARN resource management logic assumes resource allocated to a container
is fixed during the lifetime of it. When users want to change a resource 
> of an allocated container the only way is releasing it and allocating a new container
with expected size.
> Allowing run-time changing resources of an allocated container will give us better control
of resource usage in application side



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message