hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Steve Loughran (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5670) Hadoop configurations should be read from a distributed system
Date Thu, 23 Apr 2009 13:53:30 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12701937#action_12701937
] 

Steve Loughran commented on HADOOP-5670:
----------------------------------------

Looking at my subclassed configuration code [http://smartfrog.svn.sourceforge.net/viewvc/smartfrog/trunk/core/components/hadoop/src/org/smartfrog/services/hadoop/conf/ManagedConfiguration.java?view=markup]
I am reminded of some things

# It subclasses {{JobConf}}, as some things expect to be handed a JobConf
# In the past, I did have it *live*, bound to the SmartFrog configuration graph, so (until
serialized) every change in the SF configuration was reflected in the result of the get()
method. 
# That was pretty complex, I had to subclass a lot of methods that assumed that all state
was stored statically in two properties files.
# It proved easier to comment the overrides out and move to grabbing all the values once,
and no longer being live. 

Some people have mentioned that they'd like daemons to re-read their config more dynamically.
That can get very confusing, as you may have some config values (ports, network cards, temp
directories) whose changes only get picked up by a full restart. There are two tactics we've
used in the past for this
# Have your service support a "reload" operation which loads in all values and resets thing
# Just restart the service
Provided the cost of restarting is low, restarting services is way, way easier. 

> Hadoop configurations should be read from a distributed system
> --------------------------------------------------------------
>
>                 Key: HADOOP-5670
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5670
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: conf
>            Reporter: Allen Wittenauer
>
> Rather than distributing the hadoop configuration files to every data node, compute node,
etc, Hadoop should be able to read configuration information (dynamically!) from LDAP, ZooKeeper,
whatever.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message