incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matthieu Morel <mmo...@apache.org>
Subject Re: [Discuss] Runtime Isolation for s4?
Date Tue, 01 May 2012 19:08:47 GMT
On 4/29/12 2:49 AM, Nan N. Jiang wrote:
> Hi Team,
>
> I want to start a discuss about ‘Runtime isolation’ for s4.
>
> Currently, in S4 and S4-piper, following S4’s symmetric principal, all applications
will deployed in all nodes.
> And when starting s4 service, all applications deployed will run in the same JVM.
> This can be a disaster.
> Different application will fight for the JVM/hardware resource.
> Bad application even could kill other applications.
>
> Yes. Application Isolation could release the pain.
> But all applications are still fight for the resource in the same node.
> And all the PE in the same Application still fight for JVM resource.
>
> Thinking about a big application which contains hundreds of PE prototypes,
> If we want start this application in one JVM,
> Our s4 cluster must have a set of very high performance machines due to the CPU/Memory
and other requirement of this application.
> This will limit the usage of S4 in big solution.
> I think the future of s4 cluster will be a big set of normal boxes instead of a small
set of high performance boxes.
>
> So, compare application isolation, could we go further?
> Could we provide Runtime Isolation function for S4?
> Could we change the local deploy unit from application to a group of PE? even one PE?
> If this function can be provided by s4,
> Every PE could run in it’s own JVM if needed.
> Then the resource fighting between PE will be minimum.

Deploying different PEs to different nodes would break the symmetry 
which is a key property of the design of S4. However, you are pointing 
out a real issue, and one way to provide PE isolation might be at the 
level of partitioning. We could provide a way to provide exclusive 
partitioning for some PEs using a partition exclusiveness scheme, which 
would be propagated to every node. Concretely, messages for PE with key 
K (or a set of keys) would be sent to node N, and only messages with 
such keys would be sent to node N. This way, we provide PE isolation, 
without modifying the design of the platform.

Do you think this could properly address the isolation issue?
To all: would that harm the overall design of the platform?

Thanks!

Matthieu

>
> Let’s discuss about this.
>
> Regards,
> Jiang Nan
>


Mime
View raw message