hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Allen Wittenauer <awittena...@linkedin.com>
Subject Re: Map Reduce in heterogeneous environ..
Date Thu, 11 Mar 2010 13:45:24 GMT



On 3/11/10 4:53 AM, "Segel, Mike" <msegel@navteq.com> wrote:
> Then push them out respective of their machine types?


... and this is the fundamental problem.

I'm still a big fan of using LDAP to help us solve this problem.  Rather
than push configs, read them from an LDAP store.  Roughly:

cn=mygrid
 .. define common configs

cn=datanode1,cn=mygrid
... datanode properties
hosts: hostname1,hostname2, @netgroup1, ...   Lists of hosts/netgroups this
set of properties applies to


cn=datanode2,
... datanode properties
hosts:hostname4, hostname5, @netgroup2 ... 2nd set of hosts

Etc.

Pushing this is part of the problem. If Hadoop was smart enough to read its
configs from a central store such as LDAP, it would make things much much
much easier.

See https://issues.apache.org/jira/browse/HADOOP-5670 for more discussion.


Mime
View raw message