hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1197) Support increasing resources of an allocated container
Date Mon, 16 Sep 2013 09:03:53 GMT

    [ https://issues.apache.org/jira/browse/YARN-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13768180#comment-13768180

Wangda Tan commented on YARN-1197:

Here is the tentative plan:
To make it simple, we can start with only support increase container resource when container
state is "ACQUIRED", because when container is running, we need find a way to inform NM to
update the quato of a running process, which will affect lots of parts.
# Add a field in ResourceRequest, like set/getExistingContainerId() to indicate the resource
will be added to existing container
# When the ask-list submitted to YarnScheduler, it will be normalized:
** When existingContainerId set
*** Check if container with "existingContainerId" in state ACQUIRED, abondon this request
if container not exist OR state not ACQUIRED.
*** Check if capability of this container + asked capability > scheduler.maximum-allocation-mb,
if yes, normalize ask capability 
*** Other fields will be ignored except "capability", and the "priority/host" will use the
existing container's priority/host
*** Update resource requests in AppSchedulingInfo
# Add "increase existing container resource" support in LeafQueue of capa/fifo scheduler,
and FSLeafQueue in fair scheduler. We can treat "increase existing container resource request"
as same as other resource request except,
** Only "node-local" available
** Don't create new container
# After we successfully allocated resource to an existing container, we need refresh its container
token, send back to AM. AM can use the new container to launch processes via NM.

Any thoughts? I wish I could help to move this ahead if we can reach an agreement on a rough
idea. :)
> Support increasing resources of an allocated container
> ------------------------------------------------------
>                 Key: YARN-1197
>                 URL: https://issues.apache.org/jira/browse/YARN-1197
>             Project: Hadoop YARN
>          Issue Type: Task
>          Components: api, nodemanager, resourcemanager
>    Affects Versions: 2.1.0-beta
>            Reporter: Wangda Tan
> Currently, YARN cannot support merge several containers in one node to a big container,
which can make us incrementally ask resources, merge them to a bigger one, and launch our
processes. The user scenario is described in the comments.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message