streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Geer <ch...@cxtsoftware.com>
Subject Re: [DISCUSS] Switching Streams from Camel deployment to .war deployment
Date Thu, 31 Oct 2013 21:00:22 GMT
On Thu, Oct 31, 2013 at 1:47 PM, Matt Franklin <m.ben.franklin@gmail.com>wrote:

> >
> > Matt, I understand what you are trying to do here but again, big picture
> > I'm sure sure it's this clean cut. Maybe it is for simple implementations
> > but I would think larger system would require a combination of things
> like
> > Storm and Camel working together.
> >
> >
> Possibly.  The main point I was trying to get across is that we need a core
> way to define:
>
> * An over-arching workflow for processing
> * Boundary message formats
> * Utilities and common models
> * Interfaces for workflow steps that can be implemented via whatever
> mechanism suites the solution best
>
> In the case where you are processing millions of items a day, this
> orchestration layer is the single biggest value add of the project IMO.
>

I agree. My opinion is that the message formats (xml/json - not java
interface) are the most important piece. If that can be nailed down the
rest will come together.

For example we know that the system should be able to ingest and output in
the activitystrea.ms format for starters. But we also wanted to handle
things like aggregation, so what does an aggregate message look like? What
other message formats do we need to consider?

Chris

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message