hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alejandro Abdelnur (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1197) Support changing resources of an allocated container
Date Thu, 03 Oct 2013 17:56:44 GMT

    [ https://issues.apache.org/jira/browse/YARN-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13785403#comment-13785403
] 

Alejandro Abdelnur commented on YARN-1197:
------------------------------------------

Bikas, makes sense, thanks for summarizing.

On the decreasing. Given that we also do a round loop AM->NM->RM->AM, Why not make
it a bit more symmetric

  *AM asks RM to decrease a container
  *RM notifies NM on next heartbeat about container decreasing

With this approach the RM can enforce the MIN on AM decrease and reject it if below MIN. Also,
there is not need to notify the AM of the decrease taking place as the AM requested that.
And as it is a decrease the AM can instruct the container to shrink even if the RM does not
told the NM yet. Furthermore, I would expect an AM instructs a container to shrink before
asking Yarn to avoid a race condition that could kill the container for using more resources
than it should.

Also, by doing this there would not be difference in the free resources bookkeeping in the
RM and the NMs. Thing that may be handy not to complicate things for YARN-311.

Thoughts?

> Support changing resources of an allocated container
> ----------------------------------------------------
>
>                 Key: YARN-1197
>                 URL: https://issues.apache.org/jira/browse/YARN-1197
>             Project: Hadoop YARN
>          Issue Type: Task
>          Components: api, nodemanager, resourcemanager
>    Affects Versions: 2.1.0-beta
>            Reporter: Wangda Tan
>         Attachments: yarn-1197.pdf
>
>
> Currently, YARN cannot support merge several containers in one node to a big container,
which can make us incrementally ask resources, merge them to a bigger one, and launch our
processes. The user scenario is described in the comments.



--
This message was sent by Atlassian JIRA
(v6.1#6144)

Mime
View raw message