hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5670) Hadoop configurations should be read from a distributed system
Date Tue, 16 Feb 2010 16:27:28 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12834293#action_12834293

Allen Wittenauer commented on HADOOP-5670:

Small (I'd say tiny) grids that can get away with using NFS is not the target audience here.

Besides being a SPOF, for large installations having your entire grid and your clients mounting
the same fs is likely impossible due to firewall restrictions.   Using a distributed system
such as LDAP or ZK or whatever means you also have a higher chance of replicating configs
across firewall boundaries in a secure fashion.  

Also, if Hadoop is deployed to optimize certain features, chances are very very high that
a site is going to have multiple configs for the same grid.  For example, removing certain
non-client related options out of mapred-site.xml or specifically tuning the HDFS audit log.

> Hadoop configurations should be read from a distributed system
> --------------------------------------------------------------
>                 Key: HADOOP-5670
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5670
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: conf
>            Reporter: Allen Wittenauer
> Rather than distributing the hadoop configuration files to every data node, compute node,
etc, Hadoop should be able to read configuration information (dynamically!) from LDAP, ZooKeeper,

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message