incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Frank Zheng <>
Subject Re: S4 Load Balancing Mechanism
Date Fri, 12 Oct 2012 01:36:40 GMT

Thanks for your reply.
So the dynamic load balancing is not implemented yet, right?
In case of static load balancing, if there are two working PE node without
any standby nodes,
when one fails, does the workload for the remaining working PE increase?

Besides, can zeno prototype do dynamic load balancing?


On Thu, Oct 11, 2012 at 7:00 PM, Matthieu Morel <> wrote:

> On 10/11/12 12:35 PM, Daniel Gómez Ferro wrote:
>> On Thu Oct 11 05:52:01 2012, Frank Zheng wrote:
>>> Hi All,
>>> I want to figure out the mechanism of load balancing in S4.
>>> In the paper and web site, it says ZooKeeper takes this
>>> responsibility, but the description is not very clear.
>>> I want to figure out that how ZooKeeper does load balancing.
>>> For example, if there are two working PE nodes, and 120 messages per
>>> second. The adapter distributes these messages evenly, 60 per PE node,
>>> right?
>>> When one PE node is added into the cluster, does it mean ZooKeeper
>>> balances the messages, 40 per PE node?
>>> And how does ZooKeeper decide which message to be distributed to which
>>> node?
>> Hi,
>> Currently load balancing is static. When you define your cluster you
>> specify the number of tasks (say, 2) which won't change for this
>> cluster. If you have more than 2 nodes, the rest will be backup nodes.
>> In case one of the serving nodes fails, a backup node will pick up the
>> orphaned task and start serving requests, but the load will always be
>> spread among the 2 tasks that you defined at the beginning.
>> Having dynamic load balancing would be really cool, but I don't know if
>> there are currently any efforts in that direction.
> There is the zeno prototype , which is
> actually independent of S4. It also relies on ZooKeeper for metadata about
> partitions.
> Matthieu

Zheng Yu
Mobile:  (852) 60670059

View raw message