hadoop-zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maarten Koopmans <maar...@vrijheid.net>
Subject Re: node symlinks
Date Sat, 24 Jul 2010 21:39:48 GMT
Yes, I thought about Cassandra or Voldemort, but I need ZKs guarantees 
as it will provide the file system hierarchy to a flat object store so I 
need locking primitives and consistency. Doing that on top of Voldemort 
will give me a scalable version of ZK, but just slower. Might as well 
find a way to scale across ZK clusters.

Also, I want to be able to add clusters as the number of nodes grows. 
Note that the #nodes will grow with the #users of the system, so the 
clusters can grow sequentially, hence the symlink idea.

--Maarten

On 07/24/2010 11:12 PM, Ted Dunning wrote:
> Depending on your application, it might be good to simply hash the node name
> to decide which ZK cluster to put it on.
>
> Also, a scalable key value store like Voldemort or Cassandra might be more
> appropriate for your application.  Unless you need the hard-core guarantees
> of ZK, they can be better for large scale storage.
>
> On Sat, Jul 24, 2010 at 7:30 AM, Maarten Koopmans<maarten@vrijheid.net>wrote:
>
>> Hi,
>>
>> I have a number of nodes that will grow larger than one cluster can hold,
>> so I am looking for a way to efficiently stack clusters. One way is to have
>> a zookeeper node "symlink" to another cluster.
>>
>> Has anybody ever done that and some tips, or alternative approaches?
>> Currently I use Scala, and traverse zookeeper trees by proper tail
>> recursion, so adapting the tail recursion to process "symlinks" would be my
>> approach.
>>
>> Bst, Maarten
>>
>


Mime
View raw message