Mailing-List: contact yarn-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Date: Thu, 20 Jul 2017 17:21:00 +0000 (UTC)
From: "Wangda Tan (JIRA)" <jira@apache.org>
To: yarn-issues@hadoop.apache.org
Message-ID: <JIRA.13088005.1500402455000.294290.1500571260143@Atlassian.JIRA>
In-Reply-To: <JIRA.13088005.1500402455000@Atlassian.JIRA>
References: <JIRA.13088005.1500402455000@Atlassian.JIRA> <JIRA.13088005.1500402455185@jira-lw-us.apache.org>
Subject: [jira] [Commented] (YARN-6840) Leverage RMStateStore to store
 scheduler configuration updates
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 20 Jul 2017 17:21:05 -0000


    [ https://issues.apache.org/jira/browse/YARN-6840?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16095022#comment-16095022 ] 

Wangda Tan commented on YARN-6840:
----------------------------------

[~Naganarasimha], I think storing to a single zk node should be fine. Not sure if you could help to do an experiment: what is the size of the capacity-scheduler.xml with several hundreds queues after compression? I expect it should be less than 30kb. And as number of queue grows, compression rate should increase as well (more duplicated fields). We can also store diff of changes instead of store whole config file for every update.

> Leverage RMStateStore to store scheduler configuration updates
> --------------------------------------------------------------
>
>                 Key: YARN-6840
>                 URL: https://issues.apache.org/jira/browse/YARN-6840
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Wangda Tan
>
> With this change, user doesn't have to setup separate storage system (like LevelDB) to store updates of scheduler configs. And dynamic queue can be used when RM HA is enabled.


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

---------------------------------------------------------------------
To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org