zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jordan Zimmerman <jzimmer...@netflix.com>
Subject Re: Use cases for ZooKeeper
Date Thu, 05 Jan 2012 08:11:03 GMT
OK - so this is two options for doing the same thing. You use a Leader
Election algorithm to make sure that only one node in the cluster is
operating on a work unit. Curator has an implementation (it's really just
a distributed lock with a slightly different API).


On 1/5/12 12:04 AM, "Josh Stone" <pacesysjosh@gmail.com> wrote:

>Thanks for the response. Comments below:
>On Wed, Jan 4, 2012 at 10:46 PM, Jordan Zimmerman
>> Hi Josh,
>> >Second use case: Distributed locking
>> This is one of the most common uses of ZooKeeper. There are many
>> implementations - one included with the ZK distro. Also, there is
>> https://github.com/Netflix/curator
>> >First use case: Distributing work to a cluster of nodes
>> This sounds feasible. If you give more details I and others on this list
>> can help more.
>Sure. I basically want to handle race conditions where two commands that
>operate on the same data are received by my cluster of znodes,
>concurrently. One approach is to lock on the data that is effected by the
>command (distributed lock). Another approach is make sure that all of the
>commands that operate on any set of data are routed to the same node,
>they can be processed serially using local synchronization. Consistent
>hashing is an algorithm that can be used to select a node to handle a
>message (where the inputs are the key to hash and the number of nodes in
>the cluster).
>There are various implementations for this floating around. I'm just
>interesting to know how this is working for anyone else.
>> -JZ
>> ________________________________________
>> From: Josh Stone [pacesysjosh@gmail.com]
>> Sent: Wednesday, January 04, 2012 8:09 PM
>> To: user@zookeeper.apache.org
>> Subject: Use cases for ZooKeeper
>> I have a few use cases that I'm wondering if ZooKeeper would be suitable
>> for and would appreciate some feedback.
>> First use case: Distributing work to a cluster of nodes using consistent
>> hashing to ensure that messages of some type are consistently handled by
>> the same node. I haven't been able to find any info about ZooKeeper +
>> consistent hashing. Is anyone using it for this? A concern here would be
>> how to redistribute work as nodes come and go from the cluster.
>> Second use case: Distributed locking. I noticed that there's a recipe
>> this on the ZooKeeper wiki. Is anyone doing this? Any issues? One
>> would be how to handle orphaned locks if a node that obtained a lock
>> down.
>> Third use case: Fault tolerance. If we utilized ZooKeeper to distribute
>> messages to workers, can it be made to handle a node going down by
>> re-distributing the work to another node (perhaps messages that are not
>> ack'ed within a timeout are resent)?
>> Cheers,
>> Josh

View raw message