hadoop-hdfs-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jonathan Hung <jyhung2...@gmail.com>
Subject [DISCUSS] Merging API-based scheduler configuration to trunk/branch-2
Date Thu, 28 Sep 2017 18:32:29 GMT
Hi everyone,

Starting this thread to discuss merging API-based scheduler configuration
to trunk/branch-2. The feature adds the framework for allowing users to
modify scheduler configuration via REST or CLI using a configurable backend
(leveldb/zk are currently supported), and adds capacity scheduler support
for this. The umbrella JIRA is YARN-5734. All the required work for this
feature is done and committed to branch YARN-5734, and a full diff has been
generated at YARN-7241.

Regarding compatibility, this feature is configurable and turned off by
default.

The feature has been tested locally on a couple RMs (since it is an RM only
change), with queue addition/removal/updates tested on single RM (leveldb)
and two RMs (zk). Also we verified the original configuration update
mechanism (via refreshQueues) is unaffected when the feature is off/not
configured.

Our original plan was to merge this to trunk (which is what the YARN-7241
diff is based on), and port to branch-2 before the 2.9 release. @Andrew,
what are your thoughts on also merging this to branch-3.0?

Thanks!

Jonathan Hung

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message