streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jason Letourneau <jletournea...@gmail.com>
Subject Re: Using Camel for Routing
Date Wed, 16 Oct 2013 22:49:02 GMT
I think it depends on how you define 'no problem' - deployment
infrastructure is a big part of that - messaging provides more flexibility
in deployment and capability additions at different layers of the
application vs a single web app and a shotgun approach to evolving it

I agree it should still be on the table as we define what is most important
to this community

On Wednesday, October 16, 2013, Danny Sullivan wrote:

> I think if we sacrifice scalability in the long run by switching from
> Camel to a web service then we should absolutely continue to use messaging
> and Camel. I'm confident, however, that applications exist that have a web
> service based model that service millions of requests and have no problem
> scaling without the help of Camel or any other messaging framework.
> I'm going to keep the web-service branch separate for now, but I'd still
> like to keep the option on the table until we explore building scalable
> applications without the use of Camel. Though I can see the benefits of
> using Camel, I think the complexity it adds makes it worth taking a look at
> other options.
> -Danny
>
> > Date: Wed, 16 Oct 2013 16:26:43 -0400
> > Subject: Re: Using Camel for Routing
> > From: jletourneau80@gmail.com
> > To: dev@streams.incubator.apache.org
> >
> > Yeah the more I think through this in terms of implications of a sole
> > web app - I am skepticle that it scales to interesting use cases.  I
> > am not interested in streams to monitor "a few" facebook friends, I am
> > interested in streams to monitor millions of events and activities
> > generated from web site activity (clicks or tweets whatever) or
> > enterprise assets across the globe.
> >
> > On Wed, Oct 16, 2013 at 4:15 PM, Chris Geer <chris@cxtsoftware.com>
> wrote:
> > > On Wed, Oct 16, 2013 at 11:58 AM, Danny Sullivan <
> dsullivan7@hotmail.com>wrote:
> > >
> > >> I see the value in Streams as the ability of subscribers to get
> activity
> > >> based on filters.
> > >
> > >
> > > I agree
> > >
> > >
> > >> To be clear, I do not want to eliminate processing, but do want to
> make
> > >> the focus of processing tailored to the subscribers' filters. My more
> grand
> > >> vision is that only the most relevant activity will be returned to the
> > >> Subscriber and similar activity will be aggregated into single
> activities
> > >> (this would be a little down the road). This processing would happen
> after
> > >> the activity is stored in the DB.
> > >>
> > >
> > > This brings me back to the concern about having it all within one web
> app.
> > > If the webapp accepts inputs, aggregates/filters it and manages it's
> own
> > > subscribers, how you do scale it, or provide redundancy? The beauty of
> > > using a message based system (i.e. Camel...along with Storm possibly)
> is
> > > that it allows you to partition the application however you want. So
> if you
> > > want to have your ingest web service running on one machine and your
> > > subscriber service running somewhere else and a third machine for your
> > > really resource intensive aggregation component, you can. With the web
> app
> > > you could do vertical scaling on one machine, or load balancing of all
> > > components across multiple machines but it doesn't give you the save
> > > flexibility.
> > >
> > >
> > >> Do you imagine that Streams will come packaged with some processing
> > >> components that will format other non-activitystrea.ms formatted
> json? I
> > >> do think it'd be pretty cool if you could fire up Streams, input a
> couple
> > >> Facebook Friends and people you follow on Twitter, and have all of
> that
> > >> activity posted in one place. That might be better suited for a new
> > >> separate module (something like streams-format?) if we decide on the
> > >> web-application route.
> > >>
> > >
> > > I think it has to come with something to entice more users. If
> everyone has
> > > to build their own translation to even run "Hello World" it will be a
> > > larger barrier to entry.
> > >
> > >
> > >> -Danny
> > >>
> > >> > Date: Wed, 16 Oct 2013 11:22:51 -0700
> > >> > Subject: Re: Using Camel for Routing
> > >> > From: chris@cxtsoftware.com
> > >> > To: dev@streams.incubator.apache.org
> > >> >
> > >> > On Wed, Oct 16, 2013 at 11:12 AM, Danny Sullivan <
> dsullivan7@hotmail.com
> > >> >wrote:
> > >> >
> > >> > > I think what is happening here is that we have different ideas
> about
> > >> the
> > >> > > architecture of the application.
> > >> > > Here's what I'm thinking:
> > >> > > *All processing of activity would happen BEFORE it's posted to
> > >> Streams*All
> > >> > > activity posted to Streams is in activitystrea.ms format
> > >> > > The Camel/EIP/OSGi way:
> > >>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message