hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-6840) Leverage RMStateStore to store scheduler configuration updates
Date Thu, 20 Jul 2017 17:21:00 GMT

    [ https://issues.apache.org/jira/browse/YARN-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095022#comment-16095022
] 

Wangda Tan commented on YARN-6840:
----------------------------------

[~Naganarasimha], I think storing to a single zk node should be fine. Not sure if you could
help to do an experiment: what is the size of the capacity-scheduler.xml with several hundreds
queues after compression? I expect it should be less than 30kb. And as number of queue grows,
compression rate should increase as well (more duplicated fields). We can also store diff
of changes instead of store whole config file for every update.

> Leverage RMStateStore to store scheduler configuration updates
> --------------------------------------------------------------
>
>                 Key: YARN-6840
>                 URL: https://issues.apache.org/jira/browse/YARN-6840
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Wangda Tan
>
> With this change, user doesn't have to setup separate storage system (like LevelDB) to
store updates of scheduler configs. And dynamic queue can be used when RM HA is enabled.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org


Mime
View raw message