hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Edward Capriolo (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5670) Hadoop configurations should be read from a distributed system
Date Thu, 29 Oct 2009 01:58:59 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12771257#action_12771257

Edward Capriolo commented on HADOOP-5670:

I like the way glusterfs handles this.


On the glusterfs server it holds all the client configurations in a configuration file. The
cluster client can either be started up with a local configuration file or be given an IP/PORT
and it will download its configuration from the server. 

Datanode should probably pull it its configuration from the namenode.
Tasktracker should pull its configuration from jobtracker

On the practicality side, large numbers of server will likely share a configuration. We should
have a concept of a default configuration, and then per hosts overrides, and a way of saying
groups of servers all share the same configuration file. 

I think with reloads a deamon like a DataNode could theoretically check the configuration
and then make its own decision on if could apply the change, or if it should restart because
of the change. 

> Hadoop configurations should be read from a distributed system
> --------------------------------------------------------------
>                 Key: HADOOP-5670
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5670
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: conf
>            Reporter: Allen Wittenauer
> Rather than distributing the hadoop configuration files to every data node, compute node,
etc, Hadoop should be able to read configuration information (dynamically!) from LDAP, ZooKeeper,

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message