hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bikas Saha (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1197) Support changing resources of an allocated container
Date Thu, 03 Oct 2013 07:28:51 GMT

    [ https://issues.apache.org/jira/browse/YARN-1197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13784887#comment-13784887

Bikas Saha commented on YARN-1197:

I dont think Sandy meant that the AM first tells the NM to decrease the size and then the
NM informs the RM. He meant AM asks the RM. The RM decreases/increases the size and then the
AM informs the NM about the change. RM-NM communication via heartbeat that may happen after
some time. 

For decreasing resources, if the RM is to consider the free resource available only after
the AM informs the NM and the NM heartbeats with the RM then this change may become more complicated
since the current schedulers dont expect any lag in their allocations. This will also delay
the allocation of the free space to others. Also this delay is determined by when the AM syncs
with the NM. Thats not a good property. We should probably assume the decrease to be effective
immediately and RM-NM sync should enforce that. The downside is that for the duration of the
heartbeat interval, the node may get overbooked but that should not be a problem in practice
since the container would already be using a lower value of resources before the AM asked
its capacity to be decreased.
The same problem does not hold for increasing resources.

> Support changing resources of an allocated container
> ----------------------------------------------------
>                 Key: YARN-1197
>                 URL: https://issues.apache.org/jira/browse/YARN-1197
>             Project: Hadoop YARN
>          Issue Type: Task
>          Components: api, nodemanager, resourcemanager
>    Affects Versions: 2.1.0-beta
>            Reporter: Wangda Tan
>         Attachments: yarn-1197.pdf
> Currently, YARN cannot support merge several containers in one node to a big container,
which can make us incrementally ask resources, merge them to a bigger one, and launch our
processes. The user scenario is described in the comments.

This message was sent by Atlassian JIRA

View raw message