hadoop-common-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Allen Wittenauer (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5670) Hadoop configurations should be read from a distributed system
Date Tue, 06 Apr 2010 17:53:34 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12854083#action_12854083

Allen Wittenauer commented on HADOOP-5670:

bq. it would require organizations not using ldap to run additional infrastructure to solve
a problem which could be solved by something already provided by the hadoop infrastructure.
I would feel as though if we saw many users deploying a isolated ldap installation just for
hadoop that we would have made people's life harder not easier.

If you s,LDAP,ZK,g above, you'll find it is a better fit.  The cold, hard reality is that
LDAP is everywhere, ZK is not.  There are some key features in ZK that are/were missing  in
order for it to fit here (with the good news being that those gaps are slowly closing).  But
the truth of the matter is that LDAP is a well understood technology by most IT departments
and ZK is not.  (Sidenote: it would be interesting to know how well used the security components
of ZK are...)

Also, I don't think you should think of ZK as part of the Hadoop infrastructure.  It is a
sub project (and therefore part of the ecosystem), but you can run Hadoop without using ZK
and many many many people do, including Yahoo! for years. 

bq. Allen, sorry to be a pain , but could you be a bit more pedantic about what you mean by
configurations ? Do you think a hybrid system may make sense here? 

I'll try and write/diagram something up with a concrete proposal as to how I think this should
be done, based on conversations I've had with Owen and others over the years.  You'll find
I'm thinking way beyond just a simple 10 node grid. :)

> Hadoop configurations should be read from a distributed system
> --------------------------------------------------------------
>                 Key: HADOOP-5670
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5670
>             Project: Hadoop Common
>          Issue Type: New Feature
>          Components: conf
>            Reporter: Allen Wittenauer
> Rather than distributing the hadoop configuration files to every data node, compute node,
etc, Hadoop should be able to read configuration information (dynamically!) from LDAP, ZooKeeper,

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message