hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Philip Zeyliger (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-7001) Allow configuration changes without restarting configured nodes
Date Fri, 15 Oct 2010 18:47:35 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-7001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12921493#action_12921493

Philip Zeyliger commented on HADOOP-7001:

I might be grumpy, but I think the right way to deal with configuration changes in a distributed,
fault-tolerant system is to just restart the daemon entirely.  We already have to deal with
the daemon suddenly crashing and that not affecting the system too much, so I'm wary of extra
complexity of yet another process.  In practice, many configuration variables are loaded at
start time and then stored as statics: stuff like threadpool sizes, for example.  Not to mention
that Configuration objects get copied along, so it's hard to make sure that a configuration
change propagates to all possible children.

I'll point out that the namenode and the jobtracker's fair scheduler already have mechanisms
for dynamic configuration changes.  In namenode, that's -refreshNodes.  In the jt, I think
the fair scheduler re-reads its XML configuration file on occasion.  Both of these make a
lot of sense: these are specific endpoints for managing specific data, and the semantics of
those changes are easy to understand.

-- Philip

> Allow configuration changes without restarting configured nodes
> ---------------------------------------------------------------
>                 Key: HADOOP-7001
>                 URL: https://issues.apache.org/jira/browse/HADOOP-7001
>             Project: Hadoop Common
>          Issue Type: Task
>            Reporter: Patrick Kling
> Currently, changing the configuration on a node (e.g., the name node) requires that we
restart the node. We propose a change that would allow us to make configuration changes without
restarting. Nodes that support configuration changes at run time should implement the following
> interface ChangeableConfigured extends Configured {
>    void changeConfiguration(Configuration newConf) throws ConfigurationChangeException;
> }
> The contract of changeConfiguration is as follows:
> The node will compare newConf to the existing configuration. For each configuration property
that is set to a different value than in the current configuration, the node will either adjust
its behaviour to conform to the new configuration or throw a ConfigurationChangeException
if this change is not possible at run time. If a configuration property is set in the current
configuration but is unset in newConf, the node should use its default value for this property.
After a successful invocation of changeConfiguration, the behaviour of the configured node
should be indistinguishable from the behaviour of a node that was configured with newConf
at creation.
> It should be easy to change existing nodes to implement this interface. We can start
by throwing the exception for all changes and then gradually start supporting more and more
changes at run time. (We might even consider replacing Configured with ChangeableConfigured
entirely, but I think the proposal above afford greater flexibility).

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message