incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aimee Cheng" <chengsj....@gmail.com>
Subject Review Request: Implement of runtime isolation
Date Wed, 16 Jan 2013 18:15:25 GMT

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8972/
-----------------------------------------------------------

Review request for S4 and Matthieu Morel.


Description
-------

PE runtime isolation. By setting exclusive number N for a PE, when deploying, N partition
will be allocated to this PE. And the other non-exclusive PEs will be deployed to the left
partitions as before, keep symmetrical design.

To distinguish the PE partition and the Cluster partition, use global partition to call the
partition id in a cluster. And the mainly change is in the class App, Stream and Sender.

1. In App, add a schedule method for allocating global partition to PEs

2. In Sender, change the send(String hashkey, Event event) to be send(int partition, Event
event). Because now sender cannot decide sending to which global partition only by hashkey,
PE type also need be known. I move this part of logic into Stream. Let PE decides which global
partition need be sent.

3.In Stream, add a logic for iterating PEs and checking if this event need be sent to local.


This addresses bug S4-91.
    https://issues.apache.org/jira/browse/S4-91


Diffs
-----

  subprojects/s4-base/src/main/java/org/apache/s4/base/Event.java f4186a8 
  subprojects/s4-base/src/main/java/org/apache/s4/base/Sender.java c69cc45 
  subprojects/s4-core/src/main/java/org/apache/s4/core/App.java 6b8c7ef 
  subprojects/s4-core/src/main/java/org/apache/s4/core/ProcessingElement.java 6e219a6 
  subprojects/s4-core/src/main/java/org/apache/s4/core/SenderImpl.java b103ad8 
  subprojects/s4-core/src/main/java/org/apache/s4/core/Stream.java 00137b8 
  subprojects/s4-core/src/test/java/org/apache/s4/core/ri/RuntimeIsolationTest.java PRE-CREATION

  subprojects/s4-core/src/test/java/org/apache/s4/core/ri/ScheduleTest.java PRE-CREATION 
  subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountTest.java 8ee58e5 

Diff: https://reviews.apache.org/r/8972/diff/


Testing
-------

Two unit tests for this feature, one for testing the schedule method, and the other one reuses
the WordCount test case to check the process logical.


Thanks,

Aimee Cheng


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message