hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hitesh Shah (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-1004) yarn.scheduler.minimum|maximum|increment-allocation-mb should have scheduler
Date Wed, 31 Jul 2013 18:07:49 GMT

    [ https://issues.apache.org/jira/browse/YARN-1004?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13725517#comment-13725517

Hitesh Shah commented on YARN-1004:

[~sandyr] Scheduler-specific configs are fine as long they don't affect the apis and how an
app needs to be written.

The reason I mentioned max is that max is currently exposed in the api and therefore it requires
either to be in the RM-config or an enforced config property of each scheduler impl. 

The question of max being a scheduler-specific implementation choice of whether to handle
it or not seems wrong. Based on the current api, it is a defined contract between an app and
YARN that a container greater than max will not be allocated. Having one scheduler enforce
that contract and another not enforce means that applications now need to know what scheduler
is running and change their code/run-time flow accordingly. That is a huge problem for developers
trying to write applications on YARN.

> yarn.scheduler.minimum|maximum|increment-allocation-mb should have scheduler
> ----------------------------------------------------------------------------
>                 Key: YARN-1004
>                 URL: https://issues.apache.org/jira/browse/YARN-1004
>             Project: Hadoop YARN
>          Issue Type: Improvement
>          Components: scheduler
>    Affects Versions: 2.1.0-beta
>            Reporter: Sandy Ryza
> As yarn.scheduler.minimum-allocation-mb is now a scheduler-specific configuration, and
functions differently for the Fair and Capacity schedulers, it would be less confusing for
the config names to include the scheduler names, i.e. yarn.scheduler.fair.minimum-allocation-mb,
yarn.scheduler.capacity.minimum-allocation-mb, and yarn.scheduler.fifo.minimum-allocation-mb.
> The same goes for yarn.scheduler.increment-allocation-mb, which only exists for the Fair
Scheduler, and yarn.scheduler.maximum-allocation-mb, for consistency.
> If we wish to preserve backwards compatibility, we can deprecate the old configs to the
new ones. 

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message