polygene-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jiri Jetmar <juergen.jet...@gmail.com>
Subject Re: Messaging abstraction (for 3.0)
Date Wed, 17 Jun 2015 08:54:39 GMT
Hi,

messaging is indeed one of the crucial features of a IT Platform.
I once worked for a Swiss Bank that maintained over 30k of different
Queues... :-)

I guess we are talking here about a generic internal API. There is already
a
number of messaging APIs that we can take as a basis. You mentioned already
few of them.
The issue is always the level of abstraction - high abstraction level leads
to
broad support of messaging solutions, but is not easy capable to support
special feature X.
Be we can use here a similar approach of extensions, like we did e.g. for
the spatial support.

What about to use Apache Camel as a "glue" between Zest messaging API and
the concrete messaging
implementation ?

.. my two cents ..

Cheers,
jj



2015-06-17 5:46 GMT+02:00 Niclas Hedhman <niclas@hedhman.org>:

> Gang,
> I have been thinking a bit about a "Messaging" abstraction, similar to the
> "Storage" and "Indexing" abstractions, and looked at my own implementations
> how they are done, and see if we can come up with an Extension mechanism
> that is suitable for a whole range of Messaging Extensions, such as
>
>    * Restlet
>    * Jersey
>    * JMS(?)
>    * Kafka
>    * Storm (?) [1]
>
> There are a few initial "requirements" that I have written down on my white
> board;
>
>   * UnitOfWork-like - i.e. any "result messages" are not sent until
> everything is completed successfully.
>   * "Destination" concept - the entry point for event request processing
>   * Hierarchical delegation - think "paths", "routing", "intercepts"
>   * Request Parameters - think "headers" in HTTP or other metadata
>   * Request Payload
>      * Payload objects - converted into ValueComposites through mapping
>      * Binary payloads - byte[] blob
>   * Message Receiver is created per request. Factory implementation from
> user.
>   * Optional Response Messages - probably part of a return type.
>   * Usecase metadata for Request Parameters, possibly lazy deserialization.
>   * Message Destination injection, for outgoing messages. Possibly a new
> Composite meta type.
>   * Visibility - some messaging may be internal to the application, using
> the same abstraction. So Visibility should be used as usual.
>   * Capable of distinction between Load Balanced Queues, Pub/Sub and
> Peer2Peer Queues.
>
> In very, very broad strokes, we are looking at something like this (Kafka
> used as an example);
>
>
>  ___________   +----------+   ___________
> O partition O--| Kafka ME |--O partition O
>  -----------   +----------+   -----------
>                  |      ^
>                  v      |
>       +--------------------+            +------------+
>       | Qi4j Messaging SPI | <--serves--| MC Factory |
>       +--------------------+            +------------+
>             |      ^                          |
>             v      |                          |
>      +----------------------+                 |
>      | MessageComposite (MC)|<--instantiates--+
>      +----------------------+
>
>
> This picture doesn't involve the many options required to be handled, but
> should serve as a mental model to help understand where I am coming from.
>
> Before I start looking at how this could be done, does anyone have any
> input on the topic (pun intended, as Topics are part of the problem domain
> somehow, but I haven't worked that out yet)
>
>
>
> [1] Getting an abstraction right, which allows Zest/Qi4j applications to be
> part of the Storm topology would be awesome.
> --
> Niclas Hedhman, Software Developer
> http://zest.apache.org - New Energy for Java
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message