hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sanjay Radia" <sra...@yahoo-inc.com>
Subject Re: Zeroconf for hadoop
Date Sun, 15 Feb 2009 00:04:27 GMT

----- Original Message -----
From: Ted Dunning <ted.dunning@gmail.com>
To: core-dev@hadoop.apache.org <core-dev@hadoop.apache.org>
Sent: Wed Jan 28 11:51:54 2009
Subject: Re: Zeroconf for hadoop

A big positive vote for Zookeeper.

The most salient aspect of my experience using zookeeper is that
coordination and heartbeats and discovery and failure notifications all
become nearly trivial.  The most amazing thing about ZK isn't the code that
you write, it is all the code that you never have write.

On Wed, Jan 28, 2009 at 11:01 AM, Patrick Hunt <phunt@apache.org> wrote:

> Owen O'Malley wrote:
>> On Jan 25, 2009, at 2:02 PM, nitesh bhatia wrote:
>>  Apple provides opensource discovery service called Bonjour (zeroconf).
>> I don't know enough about Zeroconf to be able to answer definitively, but
>> I suspect the hardest bit would be figuring out the approach. Of course
>> Hadoop has to continue to work on other platforms, so cross-platform
>> strategies are better.
> Take a look at ZooKeeper (a sub-project of Hadoop):
> http://hadoop.apache.org/zookeeper/
> Among other features ZooKeeper provides group membership and dynamic
> configuration support -- you could modify the various Hadoop processes to
> query & register with ZooKeeper when they come up. This could be used for
> node/service discovery as well as auto configuration of the processes.
> Patrick

Ted Dunning, CTO
4600 Bohannon Drive, Suite 220
Menlo Park, CA 94025
650-324-0110, ext. 738
858-414-0013 (m)
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message