hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Radim Kolar (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-211) Allow definition of max-active-applications per queue
Date Mon, 12 Nov 2012 15:43:13 GMT

    [ https://issues.apache.org/jira/browse/YARN-211?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13495350#comment-13495350

Radim Kolar commented on YARN-211:

I know that it is currently controlled by percent and it can be configured per queue. This
is unsuitable if you need to limit number of running applications to certain low number to
work around primitive cluster scheduling in hadoop.

For example if you have nodes with 2 GPU units, then create queue with maximum-running-tasks
1 or 2 and submit applications to that queue, there is better chance that they will not run
concurrently on one node.

Using percentage for configuration is tricky, its difficult to predict target effect and it
changes with number of nodes in cluster.
> Allow definition of max-active-applications per queue
> -----------------------------------------------------
>                 Key: YARN-211
>                 URL: https://issues.apache.org/jira/browse/YARN-211
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: capacityscheduler
>            Reporter: Radim Kolar
>         Attachments: capacity-maxactive.txt
> In some cases, automatic max-active is not enough, especially if you need less active
tasks in given queue

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message