incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aimee Cheng (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (S4-91) Runtime isolation
Date Sat, 01 Dec 2012 14:29:58 GMT

    [ https://issues.apache.org/jira/browse/S4-91?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13507975#comment-13507975
] 

Aimee Cheng edited comment on S4-91 at 12/1/12 2:29 PM:
--------------------------------------------------------

The patch I uploaded contains a feature for making PE to be exclusive. By setting exclusive
number N for a PE, when deploying, N partition will be allocated to this PE. And the other
non-exclusive PEs will be deployed to the left partitions as before, keep symmetrical design.


To distinguish the PE partition and the Cluster partition, I use global partition to call
the partition id in a cluster. And the mainly change is in App, Stream and Sender. 

1. In App, add a schedule method for allocating global partition to PEs

2. In Sender, change the send(String hashkey, Event event) to be send(int partition, Event
event). Because now sender cannot decide sending to which global partition only by hashkey,
PE type also need be known. I move this part of logic into Stream. Let PE decides which global
partition need be sent.

3.In Stream, add a logic for iterating PEs and checking if this event need be sent to local.

There are two test cases for this feature, one for testing the schedule, and another reuses
the WordCount test case to check if the logical is right. 
                
      was (Author: sjcheng):
    The patch I uploaded contains a feature for making PE to be exclusive. By setting exclusive
number N for a PE, when deploying, N partition will be allocated to this PE. And the other
non-exclusive PEs will be deployed to the left partitions as before, keep symmetrical design.


To distinguish the PE partition and the Cluster partition, I use global partition to call
the partition id in a cluster. And the mainly change is in App, Stream and Sender. 

1. In App, add a schedule method for allocating global partition to PEs

2. In Sender, change the send(String hashkey, Event event) to be send(int partition, Event
event). Because now sender cannot decide sending to which global partition only be hashkey,
PE type only need be known. I move this part of logic into Stream. Let PE decides which global
partition need be sent.

3.In Stream, add a logic for iterating PEs and checking if this event need be sent to local.

There are two test cases for this feature, one for testing the schedule, and another reuses
the WordCount test case to check if the logical is right. 
                  
> Runtime isolation
> -----------------
>
>                 Key: S4-91
>                 URL: https://issues.apache.org/jira/browse/S4-91
>             Project: Apache S4
>          Issue Type: Task
>    Affects Versions: 0.6
>            Reporter: Aimee Cheng
>            Assignee: Aimee Cheng
>         Attachments: 0001-runtime-isolation-implement.patch
>
>
> We would like to use runtime isolation. PEs from the same APP can split to several sub-sets
and run in different JVM processes, so that we might get some benefit from it, such as isolating
the PEs which cost a lot of resources, easier to find out the PE with problem. 

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message