streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Geer <ch...@cxtsoftware.com>
Subject Re: Integration with Storm and Cassandra Java Driver
Date Wed, 28 Aug 2013 20:12:30 GMT
On Wed, Aug 28, 2013 at 12:40 PM, Matt Franklin <m.ben.franklin@gmail.com>wrote:

> On Wed, Aug 28, 2013 at 2:14 PM, Danny Sullivan <dsullivan7@hotmail.com
> >wrote:
>
> > Thanks for the info Steve.
> > As I understand, Kafka would take the place of the functionality of what
> > ActiveMQ does now. Storm would take place of what Camel does now.
> >
>
> I think in the long term we need to have a flexible architecture with a few
> implementations.  The way I see it, we need collection, orchestration,
> processing pipeline, persistence and exposure.  If there is a way that we
> can define each of these components loosely coupled enough to where we
> could have a Kafka OR AMQP routing implementation that would be ideal.  I
> haven't thought through exactly how to do this myself, but wanted to offer
> that things may not be mutually exclusive.
>

I agree Matt. I've had some ideas for a while but haven't had the time to
test them out. At a high level what I think would be nice is to get rid of
the API interfaces and make it all message based. The core of the streams
server could essentially dynamically build camel routes based on the
configuration. That way you could rearrange the various processing steps
any way you see fit as long as the input/output messages were compatible.
This would also allow you to split steps across servers or even different
implementations.

Chris

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message