hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Patrick Hunt (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5670) Hadoop configurations should be read from a distributed system
Date Mon, 05 Apr 2010 20:28:27 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12853532#action_12853532
] 

Patrick Hunt commented on HADOOP-5670:
--------------------------------------

Here's a really interesting paper from Akamai related to Quorum use for configuration:
"ACMS: The Akamai Configuration Management System"
http://www.usenix.org/event/nsdi05/tech/full_papers/sherman/sherman_html/index.html

ZooKeeper has a REST based interface that might work well if client's (hadoop) primary interest
is to poll for configuration
during bootstrap. Not that one has to use REST, if the ZK client interface were used directly
then one could register
for dynamic changes via ZK watches. Additionally the hadoop nodes could take advantage of
other ZK features to implement
group membership,(nodes register themselves, and their state, with ZK, allowing simple monitoring
to be implemented), Leader election, etc...

> Hadoop configurations should be read from a distributed system
> --------------------------------------------------------------
>
>                 Key: HADOOP-5670
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5670
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: conf
>            Reporter: Allen Wittenauer
>
> Rather than distributing the hadoop configuration files to every data node, compute node,
etc, Hadoop should be able to read configuration information (dynamically!) from LDAP, ZooKeeper,
whatever.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message