incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Frank Zheng <>
Subject Re: S4 Communication Layer using ZooKeeper
Date Wed, 03 Oct 2012 06:19:25 GMT
Hi Kishore,

This describes very clearly. Thank you a lot!

Now I have another question.
When one active node dies, the standby node tries to grab the lock.
What if no standby nodes are allowed? Under this assumption, is it possible
to restart the node automatically which dies previously?


On Wed, Oct 3, 2012 at 12:51 PM, kishore g <> wrote:

> At a very high level, this is how cluster management works
> Each s4 cluster has a name space reserved /clustername in zookeeper. There
> is an initial setup process where one or many znodes are created under
> /clustername/tasks. When nodes join the cluster they check if some one has
> already claimed a task by looking at /clustername/process/, if not it grabs
> the lock by creating an ephemeral node under /clustername/process/. If all
> tasks are taken it becomes a standby node. When any active node dies, the
> standby node gets notified and tries to grab the lock.
> We can provide more details, if you can let us know which aspect of
> cluster management mechanism you are interested in.
> Thanks,
> Kishore G
> On Tue, Oct 2, 2012 at 9:17 PM, Frank Zheng <>wrote:
>> Hi All,
>> I am exploring the cluster management mechanism and fault tolerance of S4.
>> I saw that S4 used ZooKeeper in the communication layer. But it seems not
>> very clear in that pater, " S4: Distributed Stream Computing Platform".
>> I tried to search the reference "[15] Communication layer using
>> ZooKeeper, Yahoo! Inc. Tech. Rep., 2009", but it is not available.
>> Could anyone introduce me the role of ZooKeeper in S4, and the cluster
>> management mechanism in detail?
>> Thanks.
>> Sincerely,
>> Frank

Zheng Yu
Mobile:  (852) 60670059

View raw message