Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: yarn-issues@hadoop.apache.org
Date: Fri, 26 Feb 2016 23:33:18 +0000 (UTC)
From: "Arun Suresh (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.12756761.1416509285000.161604.1456529598349@Atlassian.JIRA>
In-Reply-To: <JIRA.12756761.1416509285000@Atlassian.JIRA>
References: <JIRA.12756761.1416509285000@Atlassian.JIRA>
 <JIRA.12756761.1416509285491@arcas>
Subject: [jira] [Commented] (YARN-2883) Queuing of container requests in the
 NM
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: quoted-printable


    [ https://issues.apache.org/jira/browse/YARN-2883?page=3Dcom.atlassian.=
jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=3D15170=
103#comment-15170103 ]=20

Arun Suresh commented on YARN-2883:
-----------------------------------

Thanks for the patch [~kkaranasos]. Couple of comments from my initial fly-=
by of the patch.

* Don=E2=80=99t think we should add the additional *newContainerStatus* met=
hod in ContainerStatus, since the standard pattern I see is to use the sett=
er.
* In {{Context}}, briefly give a comment about what the Key & Value of the =
maps returned by *getQueuedContainers* and *getKilledQueuedContainers*
* It may be better to create another class like {{DistributedSchedulingCont=
ext}} tha exposes both the above methods and then {{Context}} can just expo=
se {{getDistributedSchedulingContext}} which can return null if DistSched i=
s not enabled.=20
* See above comment on changes to NMContext (create a DistSchedContext and =
add both the new maps in there)
* In {{ContainerManagerImpl}}, we  should register the {{ContainerExecution=
EventType}} with the dispatcher in its contructor where most of the other d=
ispatchers are registered.. And I guess it should be there even if AMRMProx=
y is enabled or not (in which case no events will be generated)

*ContainersMonitorImpl*
* Can we replace Logical with Allocated (I am guessing that is what it mean=
s) ?
* Again, lets move all the state related to DistributedScheduling (logicalG=
uarContainers, logicalOpportContainers, logicalContainersUtilization, queue=
dGuarRequests, queuedOpportRequests and opportContainersToKill) to a separa=
te {{DistributedSchedulingState}} object.. Will be easier to reason about h=
ow the existing code has changed.
* Maybe we should mover all the right shifting ( >> 20) into a utility func=
tion inside ProcessTreeInfo ?
=09
*General Comments*
* Couple of classes have unused imports..=20
* We need some Unit tests
=09
Will provide more comments shortly..=09
=09
=09


> Queuing of container requests in the NM
> ---------------------------------------
>
>                 Key: YARN-2883
>                 URL: https://issues.apache.org/jira/browse/YARN-2883
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Konstantinos Karanasos
>            Assignee: Konstantinos Karanasos
>         Attachments: YARN-2883-yarn-2877.001.patch
>
>
> We propose to add a queue in each NM, where queueable container requests =
can be held.
> Based on the available resources in the node and the containers in the qu=
eue, the NM will decide when to allow the execution of a queued container.
> In order to ensure the instantaneous start of a guaranteed-start containe=
r, the NM may decide to pre-empt/kill running queueable containers.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)