hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jason Lowe (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-2176) CapacityScheduler loops over all running applications rather than actively requesting apps
Date Fri, 20 Jun 2014 18:31:27 GMT

    [ https://issues.apache.org/jira/browse/YARN-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14039143#comment-14039143
] 

Jason Lowe commented on YARN-2176:
----------------------------------

Ah, yes.  AppSchedulingInfo should only be created by the built-in schedulers, so we can just
have that expect the new Queue interface that has the activate/deactivate app methods.  While
we're at it we can remove the knowledge of ActiveUsersManager from AppSchedulingInfo and just
have the queues update their own ActiveUsersManager instances when their activate/deactivate
methods are called.  That will streamline the AppSchedulingInfo code a bit.

> CapacityScheduler loops over all running applications rather than actively requesting
apps
> ------------------------------------------------------------------------------------------
>
>                 Key: YARN-2176
>                 URL: https://issues.apache.org/jira/browse/YARN-2176
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacityscheduler
>    Affects Versions: 2.4.0
>            Reporter: Jason Lowe
>
> The capacity scheduler performance is primarily dominated by LeafQueue.assignContainers,
and that currently loops over all applications that are running in the queue.  It would be
more efficient if we looped over just the applications that are actively asking for resources
rather than all applications, as there could be thousands of applications running but only
a few hundred that are currently asking for resources.



--
This message was sent by Atlassian JIRA
(v6.2#6252)

Mime
View raw message