incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Leo Neumeyer <>
Subject Re: [Discuss] Runtime Isolation for s4?
Date Thu, 31 May 2012 15:25:59 GMT
To isolate resources the idea is to use different subclusters for
different applications. If you want each PE to be isolated you can
define an App with a single PE that will run on its own JVM. I'm not
sure why this would be necessary but it can be done. It would help to
see a more specific example.

On Thu, May 31, 2012 at 6:46 AM, Karthik Kambatla
<> wrote:
> Hi Nan
> I agree with you that PEs should run in isolation, and have fair resource
> allocation. Though they are very much inter-dependent, I believe these are
> two different issues we are trying to solve.
> PE Isolation:
>   1. For complete isolation of each PE, we could run each PE in a
>   different JVM. However, this comes at the expense of having an insane
>   number of PEs running on each machine in the cluster.
>   2. To avoid the large number of PEs possible by following 1, we can run
>   all PEs corresponding to one application in a different JVM. If one of the
>   PEs does crash the JVM, only PEs of the same application are affected.
> PE Resource allocation: I would categorize your example of frequent GC
> calls under this, where one PE is monopolizing the resources due to its
> application logic.
>   1. If adopt method 2 for isolation, it is the application logic that is
>   favoring one PE over the other and I think it is alright.
>   2. Dynamic load balancing (Leo's suggestion): By moving the deprived PE
>   away from the monopolizing PE, we increase the chances of fair allocation.
> Thoughts??
> Thanks
> Karthik
> On Thu, May 31, 2012 at 2:42 AM, Nan N. Jiang <>wrote:
>> Hi Flavio and Leo,
>> We are involving more and more aspects here in this topic.
>> That will make it more confused.
>> Let’s keep it sample and focus on my first question: runtime isolation.
>> After that, we can talk about other aspect.
>> For Runtime isolation,
>> One application could kill another is just one example.
>> Here, if we separate different in different jvms, then the unit of
>> isolation is still application level.
>> Actually, it’s possible that one PE could affect other PEs.
>> For example:
>> PE A has a lot of traffic, then PE A may trigger Java GC very frequently.
>> Then other PEs in the same JVM will be affected.
>> This is only one scenario that one PE will affect other PE.
>> There should be more because all these PEs are sharing the same env.
>> So, why won’t we think about to have a PE level isolation?
>> About the dynamic load balancing suggested by Leo, I think it’s more about
>> how to do the partition.
>> So this should not satisfy my use case.
>> Regards,
>> -Nan
>> On 5/30/12 5:33 AM, "Flavio Junqueira" <> wrote:
>> Check the excerpt from Nan's message down the thread.The main concern
>> seems to be the consequences of sharing a jvm. It's unclear to me if load
>> balancing alone would solve all the issues he is bringing up, although
>> resource utilization seems to be an important issue. It called my attention
>> the comment about an application killing another in the same jvm, which I'm
>> not sure we can prevent in a practical manner. I'm particularly concerned
>> about having to react aggressively and causing instabilities.
>> Nan, Could you comment, please? if we provide dynamic load balancing as
>> Leo suggests, would it satisfy your use cases?
>> -Flavio
>> On May 29, 2012, at 11:12 PM, Leo Neumeyer wrote:
>> when starting s4 service, all applications deployed will run in the
>> same JVM.
>> This can be a disaster.
>> Different application will fight for the JVM/hardware resource.
>> Bad application even could kill other applications.
>> Yes. Application Isolation could release the pain.
>> But all applications are still fight for the resource in the same node.
>> And all the PE in the same Application still fight for JVM resource.


Leo Neumeyer (@leoneu)

View raw message