Mailing-List: contact dev-help@zest.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@zest.apache.org
MIME-Version: 1.0
Sender: hedhman@gmail.com
From: Niclas Hedhman <niclas@hedhman.org>
Date: Wed, 17 Jun 2015 11:46:29 +0800
Message-ID: 
 <CADmm+KfDc37pfO3c3je1kjxL+2cU=Dk0uJhTAzifJj8fB7+Kiw@mail.gmail.com>
Subject: Messaging abstraction (for 3.0)
To: dev <dev@zest.apache.org>
Content-Type: multipart/alternative; boundary=001a113bb0d87141800518ae8a59

--001a113bb0d87141800518ae8a59
Content-Type: text/plain; charset=UTF-8

Gang,
I have been thinking a bit about a "Messaging" abstraction, similar to the
"Storage" and "Indexing" abstractions, and looked at my own implementations
how they are done, and see if we can come up with an Extension mechanism
that is suitable for a whole range of Messaging Extensions, such as

   * Restlet
   * Jersey
   * JMS(?)
   * Kafka
   * Storm (?) [1]

There are a few initial "requirements" that I have written down on my white
board;

  * UnitOfWork-like - i.e. any "result messages" are not sent until
everything is completed successfully.
  * "Destination" concept - the entry point for event request processing
  * Hierarchical delegation - think "paths", "routing", "intercepts"
  * Request Parameters - think "headers" in HTTP or other metadata
  * Request Payload
     * Payload objects - converted into ValueComposites through mapping
     * Binary payloads - byte[] blob
  * Message Receiver is created per request. Factory implementation from
user.
  * Optional Response Messages - probably part of a return type.
  * Usecase metadata for Request Parameters, possibly lazy deserialization.
  * Message Destination injection, for outgoing messages. Possibly a new
Composite meta type.
  * Visibility - some messaging may be internal to the application, using
the same abstraction. So Visibility should be used as usual.
  * Capable of distinction between Load Balanced Queues, Pub/Sub and
Peer2Peer Queues.

In very, very broad strokes, we are looking at something like this (Kafka
used as an example);


 ___________   +----------+   ___________
O partition O--| Kafka ME |--O partition O
 -----------   +----------+   -----------
                 |      ^
                 v      |
      +--------------------+            +------------+
      | Qi4j Messaging SPI | <--serves--| MC Factory |
      +--------------------+            +------------+
            |      ^                          |
            v      |                          |
     +----------------------+                 |
     | MessageComposite (MC)|<--instantiates--+
     +----------------------+


This picture doesn't involve the many options required to be handled, but
should serve as a mental model to help understand where I am coming from.

Before I start looking at how this could be done, does anyone have any
input on the topic (pun intended, as Topics are part of the problem domain
somehow, but I haven't worked that out yet)


[1] Getting an abstraction right, which allows Zest/Qi4j applications to be
part of the Storm topology would be awesome.
-- 
Niclas Hedhman, Software Developer
http://zest.apache.org - New Energy for Java

--001a113bb0d87141800518ae8a59--