incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Daniel Gómez Ferro <>
Subject Re: S4 Load Balancing Mechanism
Date Thu, 11 Oct 2012 10:35:20 GMT

On Thu Oct 11 05:52:01 2012, Frank Zheng wrote:
> Hi All,
> I want to figure out the mechanism of load balancing in S4.
> In the paper and web site, it says ZooKeeper takes this
> responsibility, but the description is not very clear.
> I want to figure out that how ZooKeeper does load balancing.
> For example, if there are two working PE nodes, and 120 messages per
> second. The adapter distributes these messages evenly, 60 per PE node,
> right?
> When one PE node is added into the cluster, does it mean ZooKeeper
> balances the messages, 40 per PE node?
> And how does ZooKeeper decide which message to be distributed to which
> node?


Currently load balancing is static. When you define your cluster you 
specify the number of tasks (say, 2) which won't change for this 
cluster. If you have more than 2 nodes, the rest will be backup nodes. 
In case one of the serving nodes fails, a backup node will pick up the 
orphaned task and start serving requests, but the load will always be 
spread among the 2 tasks that you defined at the beginning.

Having dynamic load balancing would be really cool, but I don't know if 
there are currently any efforts in that direction.

> Thanks.
> Yu

View raw message