hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1197) Support increasing resources of an allocated container
Date Tue, 17 Sep 2013 06:34:53 GMT

    [ https://issues.apache.org/jira/browse/YARN-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13769256#comment-13769256

Wangda Tan commented on YARN-1197:

Increasing resources for a container while in acquired state is not different from waiting
for some more time on the RM and allocating the larger container in the first attempt, right?
I think there's a little difference here, because waiting for resource for a big container
in the first attempt, scheduler will put the the request to "reservedContainer" at FSSchedulableNode
or FiCaSchedulerNode. This will be considered as an exception, RM will try to satisfy such
reserved container first when many different requests existed at the same time in a same node.
But if we try to ask more resource in an acquired container, I don't know what's your preferring,
do you want to create another "exception" which can put an "acquired container" to *ScheduableNode
to make it can get prior proceeded or just simply make the request as a normal resource request?

Also, the RM starts a timer for each acquired container and expects the container to be launched
on the NM before the timer expires. So we dont have too much time for the container to be
launched and thus we cannot wait for increasing the resources.
I don't know if we can refresh(receivePing) the timer for a container when we successfully
increased resource for it?

To be useful, we have to be able to increase the resources of a running container. I agree
that its a significant change. So making the change will need a more thorough investigation
and clear design proposal.
Agree! I'd like to help moving this forward, I need investigate and consider end-to-end cases
and draft a design proposal for it, once I've some ideas or question, I will let you know

> Support increasing resources of an allocated container
> ------------------------------------------------------
>                 Key: YARN-1197
>                 URL: https://issues.apache.org/jira/browse/YARN-1197
>             Project: Hadoop YARN
>          Issue Type: Task
>          Components: api, nodemanager, resourcemanager
>    Affects Versions: 2.1.0-beta
>            Reporter: Wangda Tan
> Currently, YARN cannot support merge several containers in one node to a big container,
which can make us incrementally ask resources, merge them to a bigger one, and launch our
processes. The user scenario is described in the comments.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message