-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8972/
-----------------------------------------------------------
Review request for S4 and Matthieu Morel.
Description
-------
PE runtime isolation. By setting exclusive number N for a PE, when deploying, N partition
will be allocated to this PE. And the other non-exclusive PEs will be deployed to the left
partitions as before, keep symmetrical design.
To distinguish the PE partition and the Cluster partition, use global partition to call the
partition id in a cluster. And the mainly change is in the class App, Stream and Sender.
1. In App, add a schedule method for allocating global partition to PEs
2. In Sender, change the send(String hashkey, Event event) to be send(int partition, Event
event). Because now sender cannot decide sending to which global partition only by hashkey,
PE type also need be known. I move this part of logic into Stream. Let PE decides which global
partition need be sent.
3.In Stream, add a logic for iterating PEs and checking if this event need be sent to local.
This addresses bug S4-91.
https://issues.apache.org/jira/browse/S4-91
Diffs
-----
subprojects/s4-base/src/main/java/org/apache/s4/base/Event.java f4186a8
subprojects/s4-base/src/main/java/org/apache/s4/base/Sender.java c69cc45
subprojects/s4-core/src/main/java/org/apache/s4/core/App.java 6b8c7ef
subprojects/s4-core/src/main/java/org/apache/s4/core/ProcessingElement.java 6e219a6
subprojects/s4-core/src/main/java/org/apache/s4/core/SenderImpl.java b103ad8
subprojects/s4-core/src/main/java/org/apache/s4/core/Stream.java 00137b8
subprojects/s4-core/src/test/java/org/apache/s4/core/ri/RuntimeIsolationTest.java PRE-CREATION
subprojects/s4-core/src/test/java/org/apache/s4/core/ri/ScheduleTest.java PRE-CREATION
subprojects/s4-core/src/test/java/org/apache/s4/wordcount/WordCountTest.java 8ee58e5
Diff: https://reviews.apache.org/r/8972/diff/
Testing
-------
Two unit tests for this feature, one for testing the schedule method, and the other one reuses
the WordCount test case to check the process logical.
Thanks,
Aimee Cheng
|