zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mahadev Konar <maha...@yahoo-inc.com>
Subject Re: node symlinks
Date Mon, 26 Jul 2010 16:12:31 GMT
HI Maarteen,
  Can you elaborate on your use case of ZooKeeper? We currently don't have
any symlinks feature in zookeeper. The only way to do it for you would be a
client side hash/lookup table that buckets data to different zookeeper

Or you could also store this hash/lookup table in one of the zookeeper
clusters. This lookup table can then be cached on the client side after
reading it once from zookeeper servers.


On 7/24/10 2:39 PM, "Maarten Koopmans" <maarten@vrijheid.net> wrote:

> Yes, I thought about Cassandra or Voldemort, but I need ZKs guarantees
> as it will provide the file system hierarchy to a flat object store so I
> need locking primitives and consistency. Doing that on top of Voldemort
> will give me a scalable version of ZK, but just slower. Might as well
> find a way to scale across ZK clusters.
> Also, I want to be able to add clusters as the number of nodes grows.
> Note that the #nodes will grow with the #users of the system, so the
> clusters can grow sequentially, hence the symlink idea.
> --Maarten
> On 07/24/2010 11:12 PM, Ted Dunning wrote:
>> Depending on your application, it might be good to simply hash the node name
>> to decide which ZK cluster to put it on.
>> Also, a scalable key value store like Voldemort or Cassandra might be more
>> appropriate for your application.  Unless you need the hard-core guarantees
>> of ZK, they can be better for large scale storage.
>> On Sat, Jul 24, 2010 at 7:30 AM, Maarten Koopmans<maarten@vrijheid.net>wrote:
>>> Hi,
>>> I have a number of nodes that will grow larger than one cluster can hold,
>>> so I am looking for a way to efficiently stack clusters. One way is to have
>>> a zookeeper node "symlink" to another cluster.
>>> Has anybody ever done that and some tips, or alternative approaches?
>>> Currently I use Scala, and traverse zookeeper trees by proper tail
>>> recursion, so adapting the tail recursion to process "symlinks" would be my
>>> approach.
>>> Bst, Maarten

View raw message