incubator-s4-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Leo Neumeyer <>
Subject hangout notes
Date Tue, 27 Mar 2012 17:18:24 GMT

Congrats again for the great demo using S4 across subclusters and the
PlayFramework-like command line tool to manage the cluster and deploy apps.
We are basically providing a full stack solution so users don't have to
spend time getting systems to work together.

Here is a summary of today's comments. Please let me know if you agree or
if you propose a different approach. Let's make sure we are all in sync.

  * We need to be able to extract routing info from messages and
deserialize the Event at a higher level where we know the Event Class.

  * Definition of "S4 Cluster": a collection of S4 sub-clusters.

  * Definition of "S4 Sub-cluster": A collection of nodes (each sub-cluster
has its own partitions).

  * All nodes in an "S4 Cluster" are symmetric (same code image in ALL
nodes across sub-clusters.)

  * For total isolation, send event stream to another "S4 Cluster".

  * A specific app is deployed to ALL nodes in the "S4 Cluster" but
activated on a specific "S4 Sub-cluster". This provides flexibility to
assign resources to groups of Apps while maintaining simplicity via local
method calls. (Remote invokation is also handled by the platform in the
same way and transparently.)

  * Each Node has a local collection of Sender instances, one for for each
sub-cluster in the cluster.

  * When sending an Event to an App, we use runtime tables to lookup the
sub-cluster and the corresponding Sender. Once we know the Sender, we
simply use it to send the Event using local method calls. All the
complexity is abstracted out.

  * Use EventSource to avoid application dependencies. Apps subscribe to
data sources. The event source is looked up when an App is loaded using
runtime tables.

There are additional comments in the Jira.

If you are reading this and would like to contribute to the project, please
email us! Let us know if you would like to join our weekly Google hangout
where we discuss the project.


Leo Neumeyer (@leoneu)

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message