hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2883) Queuing of container requests in the NM
Date Fri, 26 Feb 2016 23:33:18 GMT

    [ https://issues.apache.org/jira/browse/YARN-2883?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15170103#comment-15170103

Arun Suresh commented on YARN-2883:

Thanks for the patch [~kkaranasos]. Couple of comments from my initial fly-by of the patch.

* Don’t think we should add the additional *newContainerStatus* method in ContainerStatus,
since the standard pattern I see is to use the setter.
* In {{Context}}, briefly give a comment about what the Key & Value of the maps returned
by *getQueuedContainers* and *getKilledQueuedContainers*
* It may be better to create another class like {{DistributedSchedulingContext}} tha exposes
both the above methods and then {{Context}} can just expose {{getDistributedSchedulingContext}}
which can return null if DistSched is not enabled. 
* See above comment on changes to NMContext (create a DistSchedContext and add both the new
maps in there)
* In {{ContainerManagerImpl}}, we  should register the {{ContainerExecutionEventType}} with
the dispatcher in its contructor where most of the other dispatchers are registered.. And
I guess it should be there even if AMRMProxy is enabled or not (in which case no events will
be generated)

* Can we replace Logical with Allocated (I am guessing that is what it means) ?
* Again, lets move all the state related to DistributedScheduling (logicalGuarContainers,
logicalOpportContainers, logicalContainersUtilization, queuedGuarRequests, queuedOpportRequests
and opportContainersToKill) to a separate {{DistributedSchedulingState}} object.. Will be
easier to reason about how the existing code has changed.
* Maybe we should mover all the right shifting ( >> 20) into a utility function inside
ProcessTreeInfo ?
*General Comments*
* Couple of classes have unused imports.. 
* We need some Unit tests
Will provide more comments shortly..	

> Queuing of container requests in the NM
> ---------------------------------------
>                 Key: YARN-2883
>                 URL: https://issues.apache.org/jira/browse/YARN-2883
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Konstantinos Karanasos
>            Assignee: Konstantinos Karanasos
>         Attachments: YARN-2883-yarn-2877.001.patch
> We propose to add a queue in each NM, where queueable container requests can be held.
> Based on the available resources in the node and the containers in the queue, the NM
will decide when to allow the execution of a queued container.
> In order to ensure the instantaneous start of a guaranteed-start container, the NM may
decide to pre-empt/kill running queueable containers.

This message was sent by Atlassian JIRA

View raw message