incubator-s4-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthieu Morel <>
Subject Re: questions about dynamic load balancing and nodes on multiple machines
Date Fri, 24 Feb 2012 13:15:00 GMT
On 2/24/12 12:27 PM, Jia Ding wrote:
> Hi everyone,
> these days I came across S4, and the document is very helpful.
> But I'm not so clear about the dynamic load-balancing of S4, and
> Processing Nodes(PN), Processing Element(PE) running on multiple machines.
> I have some questions about these:
> 1) Can one PN run across multiple machines? e.g. one PN has 10 PEs, is
> is possible
> to run 5 PEs in machine A and other 5 PEs in machine B?

The deployment is currently symmetrical: all prototypes are deployed on 
all nodes, but PNs hold different keyed PE instances (according to the 
keys and partitioning scheme)

> 2) If the question 1's situation is possible, when machine A is busy,
> machine B is relative free,
> can migrate a PE from machine A to machine B for dynamic load-balancing?
> 3) In one PN, when a PE is overload, what S4 will do? Does S4
> automaticlly create another
> instance of this PE?
> 4) I've read the document about Zookeeper, and I know that can run S4 in
> dymanic mode.
> And I also know that S4 can run with multiple nodes and multiple machines.
> e.g. there are 3 PNs running on machine A and 1 PNs running on machine B.
> When machine A is busy, can man dynamic migrate a PN from machine A to
> machine B
> for the load-balancing.
> 5) There is one PN in machine A. machine B has no node.
> When this PN is too busy, can it be cloned and put its clone to machine B?

Load balancing is not available at the moment in the open source 
release, but there is a prototype available here



View raw message