hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrey Kuzmin (JIRA)" <j...@apache.org>
Subject [jira] Commented: (HADOOP-5670) Hadoop configurations should be read from a distributed system
Date Thu, 16 Apr 2009 20:18:14 GMT

    [ https://issues.apache.org/jira/browse/HADOOP-5670?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12699838#action_12699838

Andrey Kuzmin commented on HADOOP-5670:

> For example, assuming an HTTP plug in: it would just fetch two files and do the merge

> just like Hadoop configures things today.
Yes, this is exactly what a simplistic solution would do. I'd rather not limit myself to two
files, though: why not get ready for federated grids in advance. Further, one has to take
some care with option semantics: for instance, override of some default options may turn undesirable
or even should be prohibited. There some nits here, to summarize. 

> [For example, if I have a client machine that needs to submit jobs to two different grids,

> how can it automagically pull the proper configuration information for those two grids?
I didn't actually consider clients. If ZK supports client connections (meaning outside-world
readers and may be even writers), not sure with this - just started reading docs/code last
week, - this should be fairly straightforward. The only thing one then needs at the client
to answer your concern would be a simple "AvailableGrids" config listing respective ZK URLs
to connect to.

> Hadoop configurations should be read from a distributed system
> --------------------------------------------------------------
>                 Key: HADOOP-5670
>                 URL: https://issues.apache.org/jira/browse/HADOOP-5670
>             Project: Hadoop Core
>          Issue Type: New Feature
>          Components: conf
>            Reporter: Allen Wittenauer
> Rather than distributing the hadoop configuration files to every data node, compute node,
etc, Hadoop should be able to read configuration information (dynamically!) from LDAP, ZooKeeper,

This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message