streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Franklin <m.ben.frank...@gmail.com>
Subject Re: [DISCUSS] Switching Streams from Camel deployment to .war deployment
Date Thu, 31 Oct 2013 21:02:56 GMT
On Thu, Oct 31, 2013 at 5:00 PM, Chris Geer <chris@cxtsoftware.com> wrote:

> On Thu, Oct 31, 2013 at 1:47 PM, Matt Franklin <m.ben.franklin@gmail.com
> >wrote:
>
> > >
> > > Matt, I understand what you are trying to do here but again, big
> picture
> > > I'm sure sure it's this clean cut. Maybe it is for simple
> implementations
> > > but I would think larger system would require a combination of things
> > like
> > > Storm and Camel working together.
> > >
> > >
> > Possibly.  The main point I was trying to get across is that we need a
> core
> > way to define:
> >
> > * An over-arching workflow for processing
> > * Boundary message formats
> > * Utilities and common models
> > * Interfaces for workflow steps that can be implemented via whatever
> > mechanism suites the solution best
> >
> > In the case where you are processing millions of items a day, this
> > orchestration layer is the single biggest value add of the project IMO.
> >
>
> I agree. My opinion is that the message formats (xml/json - not java
> interface) are the most important piece. If that can be nailed down the
> rest will come together.
>
> For example we know that the system should be able to ingest and output in
> the activitystrea.ms format for starters. But we also wanted to handle
> things like aggregation, so what does an aggregate message look like? What
> other message formats do we need to consider?
>

I think a lot can be handled by the AS format using upstream duplicates or
custom extensions.  Beyond that, we need a few schemas for formats of
metrics, etc.


>
> Chris
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message