hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandy Ryza (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1241) In Fair Scheduler maxRunningApps does not work for non-leaf queues
Date Thu, 26 Sep 2013 17:01:05 GMT

    [ https://issues.apache.org/jira/browse/YARN-1241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13778967#comment-13778967
] 

Sandy Ryza commented on YARN-1241:
----------------------------------

Currently this property works by starting all apps as not runnable, and then marking some
as runnable in the update thread. Because this thread runs only every half second, app start
can be delayed by up to a half second.  The changes required to make the property work for
parent queues are a good opportunity for solving this problem as well.

I propose splitting apps in leaf queues into runnable and non-runnable lists, and keeping
a count of runnable apps in parent queues.  Determining an application's runnability at add-time
will only require looking at the size of the runnable list of the leaf queue and the numRunnable
in all the parent queues.

Then, when an application is removed, we need to check to see whether any applications can
now be made runnable.  We find the highest queue in the hierarchy that's a parent of the application's
queue and that was previously at its maxRunningApps capacity.  We go through the apps in all
leaf queues under that queue in order of start time to see if any can be made runnable.
                
> In Fair Scheduler maxRunningApps does not work for non-leaf queues
> ------------------------------------------------------------------
>
>                 Key: YARN-1241
>                 URL: https://issues.apache.org/jira/browse/YARN-1241
>             Project: Hadoop YARN
>          Issue Type: Bug
>    Affects Versions: 2.1.0-beta
>            Reporter: Sandy Ryza
>            Assignee: Sandy Ryza
>
> Setting the maxRunningApps property on a parent queue should make it that the sum of
apps in all subqueues can't exceed it

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message