hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Allen Wittenauer <awittena...@linkedin.com>
Subject Re: Map Reduce in heterogeneous environ..
Date Thu, 11 Mar 2010 13:45:24 GMT

On 3/11/10 4:53 AM, "Segel, Mike" <msegel@navteq.com> wrote:
> Then push them out respective of their machine types?

... and this is the fundamental problem.

I'm still a big fan of using LDAP to help us solve this problem.  Rather
than push configs, read them from an LDAP store.  Roughly:

 .. define common configs

... datanode properties
hosts: hostname1,hostname2, @netgroup1, ...   Lists of hosts/netgroups this
set of properties applies to

... datanode properties
hosts:hostname4, hostname5, @netgroup2 ... 2nd set of hosts


Pushing this is part of the problem. If Hadoop was smart enough to read its
configs from a central store such as LDAP, it would make things much much
much easier.

See https://issues.apache.org/jira/browse/HADOOP-5670 for more discussion.

View raw message