hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5670) Hadoop configurations should be read from a distributed system
Date Thu, 23 Apr 2009 13:53:30 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12701937#action_12701937

Steve Loughran commented on HADOOP-5670:

Looking at my subclassed configuration code [http://smartfrog.svn.sourceforge.net/viewvc/smartfrog/trunk/core/components/hadoop/src/org/smartfrog/services/hadoop/conf/ManagedConfiguration.java?view=markup]
I am reminded of some things

# It subclasses {{JobConf}}, as some things expect to be handed a JobConf
# In the past, I did have it *live*, bound to the SmartFrog configuration graph, so (until
serialized) every change in the SF configuration was reflected in the result of the get()
# That was pretty complex, I had to subclass a lot of methods that assumed that all state
was stored statically in two properties files.
# It proved easier to comment the overrides out and move to grabbing all the values once,
and no longer being live. 

Some people have mentioned that they'd like daemons to re-read their config more dynamically.
That can get very confusing, as you may have some config values (ports, network cards, temp
directories) whose changes only get picked up by a full restart. There are two tactics we've
used in the past for this
# Have your service support a "reload" operation which loads in all values and resets thing
# Just restart the service
Provided the cost of restarting is low, restarting services is way, way easier. 

> Hadoop configurations should be read from a distributed system
> --------------------------------------------------------------
>                 Key: HADOOP-5670
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5670
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: conf
>            Reporter: Allen Wittenauer
> Rather than distributing the hadoop configuration files to every data node, compute node,
etc, Hadoop should be able to read configuration information (dynamically!) from LDAP, ZooKeeper,

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message