hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Arun Suresh (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-4412) Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers
Date Fri, 08 Jan 2016 20:11:39 GMT

    [ https://issues.apache.org/jira/browse/YARN-4412?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15089859#comment-15089859

Arun Suresh commented on YARN-4412:

Thanks for the rev [~kkaranasos].

wrt 1, I understand only the wait time is relevant currently.. but in case we do need to add
more fields, it might be more effort later (also, we would need to remove the wait time field
and add the ContainerQueue object). I will remove the rest of the fields though... and I was
thinking of renaming it to 'QueuedContainersInfo'

wrt 2, Understand.. will get rid of it (Although I really like the idea of keeping it there..
with a default value of 0, so people can run tests with some randomization)

wrt 3, Oh, I actually wanted to keep it separate (the top K compute interval and corrective
mechanism calculation), since then we can have both running on different threads and different
frequencies (I was planning on having another class to compute it as part of YARN-2888)

wrt 4, Yup, it is required... Since the *DistributedSchedulingService* is going to work off
the same order of events as the scheduler, we would not want to flood it, hence the event
queue.. which is why I abstracted it out. Also I didnt see the point of opening another JIRA
just for the refactor.

wrt 5, Agreed.. good point.. will fix

wrt 6, Agreed.. will move

wrt 7, Yup.. will do

> Create ClusterMonitor to compute ordered list of preferred NMs for OPPORTUNITIC containers
> ------------------------------------------------------------------------------------------
>                 Key: YARN-4412
>                 URL: https://issues.apache.org/jira/browse/YARN-4412
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>          Components: nodemanager, resourcemanager
>            Reporter: Arun Suresh
>            Assignee: Arun Suresh
>         Attachments: YARN-4412-yarn-2877.v1.patch
> Introduce a Cluster Monitor that aggregates load information from individual Node Managers
and computes an ordered list of preferred Node managers to be used as target Nodes for OPPORTUNISTIC
container allocations. 
> This list can be pushed out to the Node Manager (specifically the AMRMProxy running on
the Node) via the Allocate Response. This will be used to make local Scheduling decisions

This message was sent by Atlassian JIRA

View raw message