hadoop-yarn-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xuan Gong <xg...@hortonworks.com>
Subject Re: [VOTE] Merge feature branch YARN-5734 (API based scheduler configuration) to trunk, branch-3.0, branch-2
Date Mon, 09 Oct 2017 16:18:52 GMT
+1 (binding)

Xuan Gong


>
>On Mon, Oct 2, 2017 at 11:09 AM, Jonathan Hung <jyhung2357@gmail.com>
>wrote:
>
>> Hi all,
>>
>> From discussion at [1], I'd like to start a vote to merge feature branch
>> YARN-5734 to trunk, branch-3.0, and branch-2. Vote will be 7 days,
>>ending
>> Monday Oct 9 at 11:00AM PDT.
>>
>> This branch adds a framework to the scheduler to allow scheduler
>> configuration mutation on the fly, including a REST and CLI interface,
>>and
>> an interface for the scheduler configuration backing store. Currently
>>the
>> capacity scheduler implements this framework.
>>
>> Umbrella is here (YARN-5734
>> <https://issues.apache.org/jira/browse/YARN-5734>), jenkins build is
>>here
>> (
>> YARN-7241 <https://issues.apache.org/jira/browse/YARN-7241>). All
>>required
>> tasks for this feature are committed. Since this feature changes RM
>>only,
>> we have tested this on a local RM setup with a suite of configuration
>> changes with no issue so far.
>>
>> Key points:
>> - The feature is turned off by default, and must be explicitly
>>configured
>> to turn on. When turned off, the behavior reverts back to the original
>>file
>> based mechanism for changing scheduler configuration (i.e. yarn rmadmin
>> -refreshQueues).
>> - The framework was designed in a way to be extendable to other
>>schedulers
>> (most notably FairScheduler).
>> - A pluggable ACL policy (YARN-5949
>> <https://issues.apache.org/jira/browse/YARN-5949>) allows admins
>> fine-grained control for who can change what configurations.
>> - The configuration storage backend is also pluggable. Currently an
>> in-memory, leveldb, and zookeeper implementation are supported.
>>
>> There were 15 subtasks completed for this feature.
>>
>> Huge thanks to everyone who helped with reviews, commits, guidance, and
>> technical discussion/design, including Carlo Curino, Xuan Gong, Subru
>> Krishnan, Min Shen, Konstantin Shvachko, Carl Steinbach, Wangda Tan,
>>Vinod
>> Kumar Vavilapalli, Suja Viswesan, Zhe Zhang, Ye Zhou.
>>
>> [1]
>> http://mail-archives.apache.org/mod_mbox/hadoop-yarn-dev/201709.mbox/%
>> 3CCAHzWLgfEAgczjcEOUCg-03ma3ROtO=PKEc9dPGGYX9rZF3NvDw@mail.gmail.com%3E
>>
>> Jonathan Hung
>>


---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-dev-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-dev-help@hadoop.apache.org


Mime
View raw message