hadoop-common-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Steve Loughran <ste...@apache.org>
Subject Re: porting HDFS to Zookeeper??
Date Wed, 02 Jul 2008 12:49:13 GMT
Sangmin Lee wrote:
> Thank you for the replies.
> I meant porting HDFS to Zookeeper to make namenode fault-tolerant and highly
> available rather than the simple usage of Zookeeper. Basically by storing
> namespace metadata in Zookeeper, we may be able to get highly available
> replicated namenodes. So I meant using Zookeeper as frontend of HDFS. Anyway
> it seems to me that there has been no actual work going on regarding this,
> isn't it? :-)
> 
> -sangmin


I think you've just volunteered.

@work we use our own tuple space thing, Anubis:
http://smartfrog.svn.sourceforge.net/viewvc/smartfrog/trunk/core/components/anubis/doc/

it lets you do the election/voting stuff to see who is in charge in an 
HA cluster, though to deal with a partition event robustly is hard; you 
need somebody that really understands the maths behind HA distributed 
computing. I am not in that category, so we are lucky there are others 
nearby who do, but it implies there is a block of code I am very 
reluctant to approach.

In fact, its a bit like security. just like it is easy to do an insecure 
system, but very hard to do a secure one, its easy to be 
low-availability, really, really hard to be HA.

-steve


Mime
View raw message