hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Miklos Szegedi (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-8250) Create another implementation of ContainerScheduler to support NM overallocation
Date Tue, 08 May 2018 22:04:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-8250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16468020#comment-16468020
] 

Miklos Szegedi commented on YARN-8250:
--------------------------------------

Thanks for the patch, [~haibochen].

ContainerScheduler could be renamed DefaultContainerScheduler to give space to extensions
later.

You could use conf.getClass in createContainerScheduler to automatically verify the parent
class.

getContainersUtilization and updateContainersUtilization might need to be synchronized or
sampled (cloned).
{code:java}
141	public ContainersMonitor getContainersMonitor() {
142	return nmContext.getContainerManager().getContainersMonitor();
143	}{code}
Usually it is considered a better practice to return nmContext and rely on the caller to retrieve
the rest.

shedQueuedOpportunisticContainers does a fifo, it might make sense to do a lifo.

 

 

> Create another implementation of ContainerScheduler to support NM overallocation
> --------------------------------------------------------------------------------
>
>                 Key: YARN-8250
>                 URL: https://issues.apache.org/jira/browse/YARN-8250
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Haibo Chen
>            Assignee: Haibo Chen
>            Priority: Major
>         Attachments: YARN-8250-YARN-1011.00.patch
>
>
> YARN-6675 adds NM over-allocation support by modifying the existing ContainerScheduler
and providing a utilizationBased resource tracker.
> However, the implementation adds a lot of complexity to ContainerScheduler, and future
tweak of over-allocation strategy based on how much containers have been launched is even
more complicated.
> As such, this Jira proposes a new ContainerScheduler that always launch guaranteed containers
immediately and queues opportunistic containers. It relies on a periodical check to launch
opportunistic containers. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message