hadoop-yarn-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Wangda Tan (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (YARN-5951) Changes to allow CapacityScheduler to use configuration store
Date Tue, 31 Jan 2017 22:52:51 GMT

    [ https://issues.apache.org/jira/browse/YARN-5951?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15847684#comment-15847684

Wangda Tan commented on YARN-5951:

Thanks [~jhung] for updating the patch, my comments/suggestions:

I think the overall configuration process will be: admin configures "use-mutation" in yarn-site.xml,
and scheduler loads implementation of conf-manager, and later uses it to get updated config
for queue refresh.

So I think to make admin can easier understand/configure it, I have following configuration
name suggestions:

"yarn.scheduler.capacity.configuration.mutation.enabled" -> "yarn.scheduler.capacity.configuration.provider"
Reason: if we provide an "enabled" option, we have to provide option for stored type. Instead
of this, I suggest to put the name of provider as acceptable values of the option, which could
be "capacity-scheduler.xml", "zk", "derby", etc.

I prefer to use the short name instead of full qualified class name to make it can be easier
used, if a specified value is not a pre-known short name, it can try to use reflection to
load the class by the given name.

And some other suggestions to code:

1) MutableConfigurationManager: Suggest to rename it to interface:CSConfigurationProvider,
which can have two methods: init and loadConfiguration.
2) Add an implementation (like FileBasedCSConfProvider), which uses existing {{CapacityScheduler#loadCapacitySchedulerConfiguration}}
to load configs. And we don't need to keep {{loadCapacitySchedulerConfiguration}} inside CapacityScheduler.
Instead, it should use provider.loadConfiguration(). 


> Changes to allow CapacityScheduler to use configuration store
> -------------------------------------------------------------
>                 Key: YARN-5951
>                 URL: https://issues.apache.org/jira/browse/YARN-5951
>             Project: Hadoop YARN
>          Issue Type: Sub-task
>            Reporter: Jonathan Hung
>            Assignee: Jonathan Hung
>         Attachments: YARN-5951-YARN-5734.001.patch
> EDIT: changing this ticket. Found that the CapacityStoreConfigurationProvider is not
necessary, since we can just grab a Configuration object from StoreConfigurationProvider with
type "SCHEDULER" and create a CapacitySchedulerConfiguration from it.
> This ticket will track changes needed for integrating other components to be used by
the capacity scheduler.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail: yarn-issues-unsubscribe@hadoop.apache.org
For additional commands, e-mail: yarn-issues-help@hadoop.apache.org

View raw message