Mailing-List: contact common-issues-help@hadoop.apache.org; run by ezmlm
Precedence: bulk
Reply-To: common-issues@hadoop.apache.org
Message-ID: <479686731.303831266339451575.JavaMail.jira@brutus.apache.org>
Date: Tue, 16 Feb 2010 16:57:31 +0000 (UTC)
From: "Edward Capriolo (JIRA)" <jira@apache.org>
To: common-issues@hadoop.apache.org
Subject: [jira] Commented: (HADOOP-5670) Hadoop configurations should be
 read from a distributed system
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/HADOOP-5670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834309#action_12834309 ] 

Edward Capriolo commented on HADOOP-5670:
-----------------------------------------

I know this is not relevant to the ticket to dynamically/remotely read configuration files but I wanted some people interested in hadoop configuration to take a look at:

http://www.jointhegrid.com/acod/

The concept is to give users a IOS like interface to configure hadoop. The configurations are generated locally and then pushed out over SSH/SCP.

The concept I am using: In most cases large portions of the configuration are shared. Their are very few exceptions to this. (IE one node has 9 disks not 8). These exceptions are done with an override list per host.

{noformat}
node6.hdfs-site.xml = shared.hdfs-site.xml + override.get(node6).hdfs-site.xml
{noformat}

Getting the effective configuration for a host is as simple as taking the shared list, and replacing the overridden variables for that host. I am using a few simple abstractions, and Java XML beans serialization for the configuration. Thus, the entire configuration is easily dumped to a single XML file, so one File object represents the state of all of the clusters configurations.

> Hadoop configurations should be read from a distributed system
> --------------------------------------------------------------
>
>                 Key: HADOOP-5670
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5670
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: conf
>            Reporter: Allen Wittenauer
>
> Rather than distributing the hadoop configuration files to every data node, compute node, etc, Hadoop should be able to read configuration information (dynamically!) from LDAP, ZooKeeper, whatever.

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.