streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Danny Sullivan <dsulliv...@hotmail.com>
Subject RE: [DISCUSS] Switching Streams from Camel deployment to .war deployment
Date Fri, 01 Nov 2013 13:56:46 GMT
Perhaps it would be best for me to take a step back and ask what problem are we trying to solve
by using Camel? My understanding is that it is to allow distribution of Streams to multiple
servers. It would be helpful to me to see another open source application that has this deployment
structure (it doesn't have to use Camel, just an application that has multiple jars communicating
with one another). Does anyone have any suggestions of software for me to look at?

> Date: Thu, 31 Oct 2013 20:23:26 -0400
> Subject: Re: [DISCUSS] Switching Streams from Camel deployment to .war deployment
> From: jletourneau80@gmail.com
> To: dev@streams.incubator.apache.org
> 
> So Danny in case it was lost in the ensuing transport discussion,
> Camel abstracts the transport, so you don't have to care if its http
> or whatever, only about the message format.  This is helpful if you
> care about plugging in components that DO care about the transport of
> course.  To answer your other questions, the downside to having the
> components all work via http (and not using messaging/Camel) is that
> everything immediately becomes point to point, which isn't a
> performance issue in its own right but limits scaling to more than one
> component being at the receiving end/fulfillment for instance as we've
> been discussing as being optimal.
> 
> On Thu, Oct 31, 2013 at 5:38 PM, Matt Franklin <m.ben.franklin@gmail.com> wrote:
> > On Thu, Oct 31, 2013 at 5:34 PM, Chris Geer <chris@cxtsoftware.com> wrote:
> >
> >> On Thu, Oct 31, 2013 at 2:32 PM, Matt Franklin <m.ben.franklin@gmail.com
> >> >wrote:
> >>
> >> > On Thu, Oct 31, 2013 at 5:28 PM, Chris Geer <chris@cxtsoftware.com>
> >> wrote:
> >> >
> >> > > On Thu, Oct 31, 2013 at 2:23 PM, Matt Franklin <
> >> m.ben.franklin@gmail.com
> >> > > >wrote:
> >> > >
> >> > > > On Thu, Oct 31, 2013 at 5:04 PM, Danny Sullivan <
> >> > dsullivan7@hotmail.com
> >> > > > >wrote:
> >> > > >
> >> > > > > I'm not quite following this, so I apologize. What I'm trying
to do
> >> > is
> >> > > > > programmatically make a request to a jar running on a separate
jvm
> >> > and
> >> > > > get
> >> > > > > the response from that call all within the same method.
Similar to
> >> > this
> >> > > > > http request:
> >> > > > >
> >> > > > > HttpGet httpget = new HttpGet();
> >> > > > >
> >> > > > > httpget.setURI(new URI("www.streams-persistence.com"));
> >> > > > >
> >> > > > > CloseableHttpResponse response = httpclient.execute(httpget);
> >> > > > >
> >> > > > > //do stuff with the response...
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > I imagine that this would translate to something in Camel
similar
> >> to
> >> > > > this:
> >> > > > >
> >> > > > >
> >> > > > > <route>
> >> > > > >
> >> > > > > <from uri="bean:subscriberService?method=getAllSubscribers"/>
> >> > > > >
> >> > > > > <inOut
> >> > > > >
> >> > > >
> >> > >
> >> >
> >> uri="addressToStreamsPersistenceJar?method=selectAllSubscribersFromDatabase"/>
> >> > > > >
> >> > > > > </route>
> >> > > > >
> >> > > >
> >> > > > Camel uses AMQP as a messaging system so that you don't have
to
> >> create
> >> > > HTTP
> >> > > > requests between sources, though it is possible to do so.  In
Storm,
> >> we
> >> > > > usually try to use a buffer system like Kafka to do the same.
> >> > > >
> >> > >
> >> > > Camel can use AMQP through ActiveMQ.
> >> > >
> >> > >
> >> > Can being the operative word.  I haven't done much with Camel.  What
> >> other
> >> > transports are available.
> >> >
> >> > OpenWire (native ActiveMQ)
> >> AMQP
> >> STOMP
> >> HTTP (can be a server or client)
> >> Mail
> >> Amazon SQS
> >> XMPP
> >> ....hundred or so more
> >>
> >> http://camel.apache.org/components.html
> >
> >
> >
> > Nice.  I was expecting RTFM, so thanks for catering to my laziness.
> >
> >
> >>
> >>
> >> >
> >> > > >
> >> > > >
> >> > > > >
> >> > > > >
> >> > > > > But it is unclear what the actual implementation would be.
> >> > > > >
> >> > > > >
> >> > > > > This actually brings me to another suggestion. Would there
be a big
> >> > > > > performance impact to have communication between the software
> >> > > components
> >> > > > > occur between http? Say the 5 software components I outlined
> >> earlier
> >> > > were
> >> > > > > packaged as 5 separate wars. These wars could communicate
with each
> >> > > other
> >> > > > > via get a post requests. This sounds unconventional offhand
so I'd
> >> > like
> >> > > > to
> >> > > > > hear some thoughts on it.
> >> > > > >
> >> > > >
> >> > > > It is most certainly possible, but IMO probably not the best
option
> >> for
> >> > > > success.  Protocols like PubSubHubbub use HTTP for a message
> >> transport.
> >> > > >
> >> > > >
> >> > > > > -Danny
> >> > > > >
> >> > > > > > Date: Thu, 31 Oct 2013 13:28:41 -0700
> >> > > > > > Subject: Re: [DISCUSS] Switching Streams from Camel
deployment to
> >> > > .war
> >> > > > > deployment
> >> > > > > > From: chris@cxtsoftware.com
> >> > > > > > To: dev@streams.incubator.apache.org
> >> > > > > >
> >> > > > > > Or Content Enricher [2]
> >> > > > > >
> >> > > > > > [2] http://camel.apache.org/content-enricher.html
> >> > > > > >
> >> > > > > >
> >> > > > > > On Thu, Oct 31, 2013 at 12:37 PM, Jason Letourneau
> >> > > > > > <jletourneau80@gmail.com>wrote:
> >> > > > > >
> >> > > > > > > check out the link here[1]
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > [1]http://camel.apache.org/request-reply.html
> >> > > > > > >
> >> > > > > > > On Thu, Oct 31, 2013 at 2:32 PM, Danny Sullivan
<
> >> > > > > dsullivan7@hotmail.com>
> >> > > > > > > wrote:
> >> > > > > > > > I have a quick Camel question that I arrived
at in the
> >> > > > > implementation of
> >> > > > > > > these new components:
> >> > > > > > > >
> >> > > > > > > > Lets say I have a method in streams-activity.jar
that needs
> >> all
> >> > > > > > > subscribers in the database. This would require
a call to the
> >> > > > > > > streams-persistence.jar. So far, I've seen camel
used mostly
> >> for
> >> > > > > passing
> >> > > > > > > data through the application, but not for making
a single
> >> > > > > request-reponse
> >> > > > > > > from within a method. How can I use Camel to get
a list of all
> >> > > > > subscribers
> >> > > > > > > in the streams-persistence.jar from the streams-activity.jar?
> >> > > > > > > >
> >> > > > > > > >> From: dsullivan7@hotmail.com
> >> > > > > > > >> To: dev@streams.incubator.apache.org
> >> > > > > > > >> Subject: RE: [DISCUSS] Switching Streams
from Camel
> >> deployment
> >> > > to
> >> > > > > .war
> >> > > > > > > deployment
> >> > > > > > > >> Date: Thu, 31 Oct 2013 08:40:44 -0400
> >> > > > > > > >>
> >> > > > > > > >> Excellent, I'll write up something as
a proof of concept and
> >> > we
> >> > > > can
> >> > > > > > > discuss further to make sure everything is vanilla.
> >> > > > > > > >>
> >> > > > > > > >> > Date: Wed, 30 Oct 2013 17:01:28
-0400
> >> > > > > > > >> > Subject: Re: [DISCUSS] Switching
Streams from Camel
> >> > deployment
> >> > > > to
> >> > > > > > > .war deployment
> >> > > > > > > >> > From: jletourneau80@gmail.com
> >> > > > > > > >> > To: dev@streams.incubator.apache.org
> >> > > > > > > >> >
> >> > > > > > > >> > That sounds pretty promising to
me.
> >> > > > > > > >> >
> >> > > > > > > >> > On Wed, Oct 30, 2013 at 2:38 PM,
Danny Sullivan <
> >> > > > > > > dsullivan7@hotmail.com> wrote:
> >> > > > > > > >> > > Thanks for the feedback. You
have an interesting point
> >> > about
> >> > > > the
> >> > > > > > > url linking to a separate processing space. Let
me tie my
> >> answer
> >> > > into
> >> > > > > your
> >> > > > > > > last question about "advocating for the simplicity
at
> >> > registration
> >> > > to
> >> > > > > give
> >> > > > > > > up flexibility at registration, but retaining
the inner "guts"
> >> of
> >> > > > > > > EIP/messaging". Consider a new architecture:
> >> > > > > > > >> > >
> >> > > > > > > >> > > streams-web.war: single entry
point to application, but
> >> > > > > functions
> >> > > > > > > ONLY as an entry point. From here Camel routes
the incoming
> >> > > requests
> >> > > > > to 4
> >> > > > > > > separate jarssubscriber-registration.jar: subscriber
> >> registration
> >> > > > > > > publisher-registration.jar: publisher registrationactivity.jar:
> >> > > > returns
> >> > > > > > > activity (also contains subscriber warehouse and
storm activity
> >> > > > > > > aggregator)publish.jar: publishes
> >> activitystreams-cassandra.jar:
> >> > > the
> >> > > > > above
> >> > > > > > > 4 jars would all have a hook into this jar which
would function
> >> > as
> >> > > a
> >> > > > > hook
> >> > > > > > > onto the database. Each jar would have camel route
output to
> >> this
> >> > > > jar.
> >> > > > > > > >> > >
> >> > > > > > > >> > > In this implementation, Camel
would no longer be the
> >> entry
> >> > > and
> >> > > > > exit
> >> > > > > > > point of a client to the application, but would
handle the
> >> > > > > communication
> >> > > > > > > between components. The flow of activity through
the
> >> application
> >> > > > would
> >> > > > > be
> >> > > > > > > method based in each jar. This would allow deployment
on up to
> >> 6
> >> > > > > different
> >> > > > > > > process spaces. However, this does not address
that there is a
> >> > > single
> >> > > > > > > server entry point, but I'm not sure if it was
a concern in the
> >> > > first
> >> > > > > place.
> >> > > > > > > >> > >
> >> > > > > > > >> > > My argument, at its basis,
is that we should move away
> >> > from
> >> > > > > using
> >> > > > > > > Camel as the entry point to the application. I
would be happy
> >> to
> >> > > > > maintain
> >> > > > > > > messaging between components.
> >> > > > > > > >> > >
> >> > > > > > > >> > >> Date: Wed, 30 Oct 2013
12:32:55 -0400
> >> > > > > > > >> > >> Subject: Re: [DISCUSS]
Switching Streams from Camel
> >> > > > deployment
> >> > > > > to
> >> > > > > > > .war deployment
> >> > > > > > > >> > >> From: jletourneau80@gmail.com
> >> > > > > > > >> > >> To: dev@streams.incubator.apache.org
> >> > > > > > > >> > >>
> >> > > > > > > >> > >> An interesting use case
that I am holding onto is the
> >> > > ability
> >> > > > > for
> >> > > > > > > >> > >> publishers to register
via a single URL (registration
> >> > > > > endpoint),
> >> > > > > > > but
> >> > > > > > > >> > >> be sent a URL back to post
to a different process space
> >> > for
> >> > > > > actual
> >> > > > > > > >> > >> publishing.  The same is
true on the subscriber front.
> >> > > > >  Currently,
> >> > > > > > > the
> >> > > > > > > >> > >> Camel/EIP infrastructure
abstracts this because
> >> different
> >> > > > > > > components
> >> > > > > > > >> > >> deployed in different process
spaces handling the route
> >> > > > > creation
> >> > > > > > > can
> >> > > > > > > >> > >> just be bolted onto a running
Streams instance without
> >> > new
> >> > > > > > > subs/pubs
> >> > > > > > > >> > >> behaving any differently
than existing.  This seems to
> >> > be a
> >> > > > > > > >> > >> potentially critical scaling
point.  Is there a way to
> >> do
> >> > > > this
> >> > > > > with
> >> > > > > > > >> > >> the Spring solution?
> >> > > > > > > >> > >>
> >> > > > > > > >> > >> The persistence point is
a good one, though I would
> >> > > classify
> >> > > > > that
> >> > > > > > > as
> >> > > > > > > >> > >> "not implemented" vs "not
possible" (not that you
> >> were).
> >> > > > > > > >> > >>
> >> > > > > > > >> > >> I'm not married to Camel,
I just like the EIP approach
> >> to
> >> > > > > building
> >> > > > > > > >> > >> something that is ultimately
a messaging system.  There
> >> > are
> >> > > > > known
> >> > > > > > > >> > >> patterns that solve at
least a subset of the problems
> >> > > Streams
> >> > > > > is
> >> > > > > > > >> > >> trying to solve and implementations
that can handle the
> >> > > load
> >> > > > > and
> >> > > > > > > I'll
> >> > > > > > > >> > >> reiterate flexibility ==
complexity almost always.
> >> > > > > > > >> > >>
> >> > > > > > > >> > >> It comes right back to
the central question: Do you
> >> want
> >> > > > > > > flexibility
> >> > > > > > > >> > >> or simplicity?  It doesn't
have to be black and white
> >> > > either
> >> > > > I
> >> > > > > > > don't
> >> > > > > > > >> > >> think...
> >> > > > > > > >> > >>
> >> > > > > > > >> > >> More pointedly: Where should
we give up flexibility for
> >> > > > > > > simplicity?  I
> >> > > > > > > >> > >> read that Danny is advocating
for the simplicity at
> >> > > > > registration to
> >> > > > > > > >> > >> give up flexibility at
registration, but retaining the
> >> > > inner
> >> > > > > > > "guts" of
> >> > > > > > > >> > >> EIP/messaging?  Thoughts?
> >> > > > > > > >> > >>
> >> > > > > > > >> > >> On Wed, Oct 30, 2013 at
11:37 AM, Danny Sullivan <
> >> > > > > > > dsullivan7@hotmail.com> wrote:
> >> > > > > > > >> > >> > My argument is not
for the IoC pattern as that can be
> >> > > (and
> >> > > > > has
> >> > > > > > > been) implemented alongside Camel. My main argument
is that the
> >> > > > syntax
> >> > > > > at
> >> > > > > > > the entry point is not only familiar but much
simpler. This
> >> > > wouldn't
> >> > > > > be a
> >> > > > > > > very strong argument if the Camel implementation
wasn't much
> >> more
> >> > > > > > > complicated but I feel that it is the case. Also,
looking
> >> toward
> >> > > the
> >> > > > > > > future, if the server is restarted, in-routes
are lost in
> >> Camel.
> >> > > The
> >> > > > > way to
> >> > > > > > > curb this is to persist the dynamic routes that
Camel creates,
> >> > and
> >> > > > > then on
> >> > > > > > > start up pull every one of these routes and recreate
a dynamic
> >> > > route
> >> > > > > for
> >> > > > > > > each one. Not only is this much easier to implement
using the
> >> > > Spring
> >> > > > > web
> >> > > > > > > implementation, but it already has been implemented
and you can
> >> > try
> >> > > > it
> >> > > > > by
> >> > > > > > > checking out the webservice branch, registering
a subscriber,
> >> > > > > restarting
> >> > > > > > > tomcat, and using the same url you had before.
This will allow
> >> > > > > subscribers
> >> > > > > > > to hang on to their urls once they register. (the
same is true
> >> > for
> >> > > > > > > publishers: you can post via the same url after
restarting
> >> > tomcat)
> >> > > > > > > >> > >> >
> >> > > > > > > >> > >> >> Date: Wed, 30
Oct 2013 11:00:21 -0400
> >> > > > > > > >> > >> >> Subject: Re: [DISCUSS]
Switching Streams from Camel
> >> > > > > deployment
> >> > > > > > > to .war deployment
> >> > > > > > > >> > >> >> From: jletourneau80@gmail.com
> >> > > > > > > >> > >> >> To: dev@streams.incubator.apache.org
> >> > > > > > > >> > >> >>
> >> > > > > > > >> > >> >> To be fair, while
the current implementation is
> >> > heavily
> >> > > > > > > camel-based,
> >> > > > > > > >> > >> >> all of the interfaces
related to Streams
> >> functionality
> >> > > are
> >> > > > > not.
> >> > > > > > >  The
> >> > > > > > > >> > >> >> current model
maps to what Matt has outlined in my
> >> > > > opinion,
> >> > > > > > > though
> >> > > > > > > >> > >> >> packing names
etc. probably don't follow that exact
> >> > > > pattern.
> >> > > > > > > >> > >> >>
> >> > > > > > > >> > >> >> With regards to
the complexity and different
> >> > components
> >> > > in
> >> > > > > the
> >> > > > > > > >> > >> >> registration process,
this was a cut at the
> >> > abstraction
> >> > > > > based
> >> > > > > > > on the
> >> > > > > > > >> > >> >> assumption that
different implementations may be
> >> > plugged
> >> > > > in
> >> > > > > and
> >> > > > > > > in
> >> > > > > > > >> > >> >> fact may live
on different processor space (ie. a
> >> > > polling
> >> > > > > > > publisher vs
> >> > > > > > > >> > >> >> a push publisher
may be instantiated on different
> >> > > servers
> >> > > > > but
> >> > > > > > > the
> >> > > > > > > >> > >> >> registration URL
is staticly defined).
> >> > > > > > > >> > >> >>
> >> > > > > > > >> > >> >> Is the main argument
I am seeing for Spring  the
> >> > > > > familiarity of
> >> > > > > > > its
> >> > > > > > > >> > >> >> IoC pattern implementation
and syntax at the entry
> >> > > point?
> >> > > > > > > >> > >> >>
> >> > > > > > > >> > >> >> On Wed, Oct 30,
2013 at 10:53 AM, Danny Sullivan <
> >> > > > > > > dsullivan7@hotmail.com> wrote:
> >> > > > > > > >> > >> >> > Could you
clarify whether the same entry points
> >> > would
> >> > > > > exist
> >> > > > > > > for the camel implementation of the core (implementing
the
> >> > > "process"
> >> > > > > > > method/ using a DynammicRouteBuilder) or would
the webservice
> >> be
> >> > > the
> >> > > > > sole
> >> > > > > > > entry point to Streams and after it enters would
it hand it off
> >> > to
> >> > > > > Camel?
> >> > > > > > > And what would be the entry point for the Storm
implementation?
> >> > > > > > > >> > >> >> > -Danny
> >> > > > > > > >> > >> >> >
> >> > > > > > > >> > >> >> >> Date:
Wed, 30 Oct 2013 10:13:04 -0400
> >> > > > > > > >> > >> >> >> Subject:
Re: [DISCUSS] Switching Streams from
> >> Camel
> >> > > > > > > deployment to .war deployment
> >> > > > > > > >> > >> >> >> From:
m.ben.franklin@gmail.com
> >> > > > > > > >> > >> >> >> To: dev@streams.incubator.apache.org
> >> > > > > > > >> > >> >> >>
> >> > > > > > > >> > >> >> >> On Tue,
Oct 29, 2013 at 2:55 PM, Danny Sullivan <
> >> > > > > > > dsullivan7@hotmail.com>wrote:
> >> > > > > > > >> > >> >> >>
> >> > > > > > > >> > >> >> >> >
My case for switching from OSGi is for
> >> simplicity
> >> > > in
> >> > > > > > > design. To follow the
> >> > > > > > > >> > >> >> >> >
path of an activity through Streams in the
> >> > > > webservice,
> >> > > > > > > there is one main
> >> > > > > > > >> > >> >> >> >
things the developer needs to understand:
> >> > > > > > > >> > >> >> >> >
The @RequestMapping annotation specifies the
> >> HTTP
> >> > > > entry
> >> > > > > > > point
> >> > > > > > > >> > >> >> >> >
There are 4 @RequestMapping annotations that
> >> > > > > correspond to
> >> > > > > > > each of the 4
> >> > > > > > > >> > >> >> >> >
ways a user enters the application:
> >> registering a
> >> > > > > > > publisher, registering a
> >> > > > > > > >> > >> >> >> >
subscriber, publishing activity, and getting an
> >> > > > > activity
> >> > > > > > > stream. Where
> >> > > > > > > >> > >> >> >> >
these are located in the source code can be
> >> found
> >> > > by
> >> > > > > > > searching for the
> >> > > > > > > >> > >> >> >> >
paths specified in the documentation (search
> >> for
> >> > > > > > > "/publisherRegister",
> >> > > > > > > >> > >> >> >> >
"/subscriberRegister", "/publishActivity",
> >> > > > > "/getActivity"
> >> > > > > > > which will all
> >> > > > > > > >> > >> >> >> >
lead you to StreamsWebController.java). From
> >> the
> >> > > > > methods
> >> > > > > > > that process
> >> > > > > > > >> > >> >> >> >
requests, the flow through the application is
> >> > > through
> >> > > > > > > methods which can be
> >> > > > > > > >> > >> >> >> >
understood by most Java programmers.
> >> > > > > > > >> > >> >> >> >
The flow of activities through the current
> >> trunk
> >> > > > > branch is
> >> > > > > > > understood as
> >> > > > > > > >> > >> >> >> >
follows:
> >> > > > > > > >> > >> >> >> >
The string "/publisher/register" (the entry
> >> point
> >> > > to
> >> > > > > > > register a publisher
> >> > > > > > > >> > >> >> >> >
specified in the documentation) is the value of
> >> > the
> >> > > > > > > >> > >> >> >> >
consumer.registrationEndpoint property defined
> >> in
> >> > > > > > > streams.propertiesThe
> >> > > > > > > >> > >> >> >> >
camelContext.xml specifies an endpoint with the
> >> > id
> >> > > > > > > >> > >> >> >> >
consumerRegistrationEndpoint to have a uri
> >> equal
> >> > to
> >> > > > the
> >> > > > > > > propertyThe
> >> > > > > > > >> > >> >> >> >
consumerRegistrationEndpoint routes from uri
> >> > > > > > > direct:publisher register with
> >> > > > > > > >> > >> >> >> >
the bean activityRegistrationProcessor nested
> >> in
> >> > > > > between
> >> > > > > > > the routeThe
> >> > > > > > > >> > >> >> >> >
streams-eip-applicationContext contains a bean
> >> > with
> >> > > > > the id
> >> > > > > > > >> > >> >> >> >
activityRegistrationProcessor created for the
> >> > class
> >> > > > > > > >> > >> >> >> >
ActivityPublisherRegistrationProcessorThe
> >> > exchange
> >> > > > will
> >> > > > > > > enter the "process"
> >> > > > > > > >> > >> >> >> >
method of the class
> >> > > > > > > ActivityPublisherRegisitrationProcessor and that
this
> >> > > > > > > >> > >> >> >> >
is because this class implements the
> >> "Processor"
> >> > > > > interface
> >> > > > > > > provided by
> >> > > > > > > >> > >> >> >> >
CamelThe direct:add-publisher-route takes the
> >> > > > exchange
> >> > > > > > > output from the
> >> > > > > > > >> > >> >> >> >
"process" method and routes it to the
> >> > > > > > > activityRegistrationProcessor
> >> > > > > > > >> > >> >> >> >
"register" method. The bean
> >> > > > > activityRegistrationProcessor
> >> > > > > > > is defined in the
> >> > > > > > > >> > >> >> >> >
streams-eip-osgi-component-import.xmlThe output
> >> > > from
> >> > > > > this
> >> > > > > > > method is then
> >> > > > > > > >> > >> >> >> >
sent to the "createNewRouteForConsumer" method
> >> of
> >> > > > > > > activityConsumerRouter.
> >> > > > > > > >> > >> >> >> >
This method creates a new route for the newly
> >> > > > > registered
> >> > > > > > > publisher using
> >> > > > > > > >> > >> >> >> >
the private static final class
> >> > > > > DynamicConsumerRouteBuilder
> >> > > > > > > which is
> >> > > > > > > >> > >> >> >> >
required to extend RouteBuilder which is
> >> provided
> >> > > by
> >> > > > > > > Camel. This
> >> > > > > > > >> > >> >> >> >
DynamicConsumerRouteBuilder contains several
> >> > > methods:
> >> > > > > > > >> > >> >> >> >
getConsumerReceiveMethod() (which corresponds
> >> to
> >> > > > @Value
> >> > > > > > > >> > >> >> >> >
${consumer.receiveMethod} which corresponds to
> >> > > > > "receive"),
> >> > > > > > > >> > >> >> >> >
getConsumerSplitMethod() (@Value
> >> > > > > > > ${consumer.splitMethod},"split"), and
> >> > > > > > > >> > >> >> >> >
getConsumerActivityQUri() (@Value
> >> > > > > ${consumerActivityQUri},
> >> > > > > > > >> > >> >> >> >
"direct:activityQ"). This is different than the
> >> > > > > > > camelContext.xml in that
> >> > > > > > > >> > >> >> >> >
the route is being created programmatically.
> >> What
> >> > > > this
> >> > > > > is
> >> > > > > > > doing is routing
> >> > > > > > > >> > >> >> >> >
input from the inroute url (which Camel does
> >> > > > > automatically
> >> > > > > > > through the
> >> > > > > > > >> > >> >> >> >
configure method which is required to be
> >> > > overridden),
> >> > > > > to
> >> > > > > > > the "receive"
> >> > > > > > > >> > >> >> >> >
method of ActivityConsumer, then to the "split"
> >> > > > method
> >> > > > > of
> >> > > > > > > ActivityConsumer,
> >> > > > > > > >> > >> >> >> >
and then to the "direct:activityQ" which if you
> >> > > look
> >> > > > > back
> >> > > > > > > in the
> >> > > > > > > >> > >> >> >> >
camelContext.xml routes to the
> >> > > > > "activemq:queue:activities"
> >> > > > > > > which then
> >> > > > > > > >> > >> >> >> >
routes to "receiveExchange"
> >> > > > > > > >> > >> >> >> >
This is the process to register a publisher.
> >> The
> >> > > > > process
> >> > > > > > > for registering a
> >> > > > > > > >> > >> >> >> >
subscriber is relatively the same though it
> >> > > involves
> >> > > > > > > separate classes with
> >> > > > > > > >> > >> >> >> >
their own private static final RouteBuilder
> >> > class.
> >> > > > > From my
> >> > > > > > > perspective, the
> >> > > > > > > >> > >> >> >> >
two most difficult things with setting this
> >> > project
> >> > > > up
> >> > > > > > > were understanding
> >> > > > > > > >> > >> >> >> >
that the "process" method of the class that
> >> > > > implements
> >> > > > > > > "Processor" is the
> >> > > > > > > >> > >> >> >> >
entry point and the DynamicConsumerRouteBuilder
> >> > > > creates
> >> > > > > > > the second entry
> >> > > > > > > >> > >> >> >> >
point (The 5th and last points). This made the
> >> > > > project
> >> > > > > > > very, VERY hard to
> >> > > > > > > >> > >> >> >> >
understand.
> >> > > > > > > >> > >> >> >> >
In addition to simplicity of design, the mvn
> >> > clean
> >> > > > > install
> >> > > > > > > of the web
> >> > > > > > > >> > >> >> >> >
service project is much faster and small scale
> >> > > > activity
> >> > > > > > > publishing is also
> >> > > > > > > >> > >> >> >> >
faster (see my email about load testing). These
> >> > are
> >> > > > > minor
> >> > > > > > > points though as
> >> > > > > > > >> > >> >> >> >
compilation has no effect on deployment. OSGi
> >> > does
> >> > > > add
> >> > > > > the
> >> > > > > > > benefit of
> >> > > > > > > >> > >> >> >> >
modularized programming which is valuable,
> >> > though I
> >> > > > > think
> >> > > > > > > the added
> >> > > > > > > >> > >> >> >> >
complexity of Camel merits moving the project
> >> > away
> >> > > > from
> >> > > > > > > this paradigm.
> >> > > > > > > >> > >> >> >> >
> >> > > > > > > >> > >> >> >>
> >> > > > > > > >> > >> >> >> I agree
that the project is pretty difficult to
> >> > > > > understand
> >> > > > > > > ATM.  I think
> >> > > > > > > >> > >> >> >> what
we need to do is think about what the
> >> > > > > responsibilities
> >> > > > > > > of the code are
> >> > > > > > > >> > >> >> >> and allow
for different implementations that are
> >> > not
> >> > > so
> >> > > > > > > tightly coupled as
> >> > > > > > > >> > >> >> >> they
are now.  For instance, having worked with
> >> > Storm
> >> > > > to
> >> > > > > > > ingest millions of
> >> > > > > > > >> > >> >> >> activities
a day, I personally would like to see
> >> > > > streams
> >> > > > > be
> >> > > > > > > responsible for
> >> > > > > > > >> > >> >> >> defining
an over-arching orchestration model that
> >> > can
> >> > > > be
> >> > > > > > > implemented within
> >> > > > > > > >> > >> >> >> a single
war or on top of a distributed system.
> >> >  This
> >> > > > > would
> >> > > > > > > look something
> >> > > > > > > >> > >> >> >> like
the follows:
> >> > > > > > > >> > >> >> >>
> >> > > > > > > >> > >> >> >> |__ Streams-Core
 (Base classes, interfaces,
> >> > > utilities,
> >> > > > > > > extensions, etc)
> >> > > > > > > >> > >> >> >> |
> >> > > > > > > >> > >> >> >> |__ Streams-Storm
(Storm implementation of the
> >> > core)
> >> > > > > > > >> > >> >> >> |
> >> > > > > > > >> > >> >> >> |__ Streams-Camel
(Camel implementation of the
> >> > core)
> >> > > > > > > >> > >> >> >> |
> >> > > > > > > >> > >> >> >> |__ Streams-WS
(Web service implementation)
> >> > > > > > > >> > >> >> >>
> >> > > > > > > >> > >> >> >>
> >> > > > > > > >> > >> >> >> >
Danny
> >> > > > > > > >> > >> >> >> >
> >> > > > > > > >> > >> >> >> >
> >> > > > > > > >> > >> >> >> >
> From: dsullivan7@hotmail.com
> >> > > > > > > >> > >> >> >> >
> To: dev@streams.incubator.apache.org
> >> > > > > > > >> > >> >> >> >
> Subject: [DISCUSS] Switching Streams from
> >> Camel
> >> > > > > > > deployment to .war
> >> > > > > > > >> > >> >> >> >
deployment
> >> > > > > > > >> > >> >> >> >
> Date: Tue, 29 Oct 2013 11:07:39 -0400
> >> > > > > > > >> > >> >> >> >
>
> >> > > > > > > >> > >> >> >> >
> The discussion thread for switching Streams
> >> > from
> >> > > > > > > Camel/osgi/Servicemix
> >> > > > > > > >> > >> >> >> >
to a single .war deployment
> >> > > > > > > >> > >> >> >> >
> >> > > > > > > >> > >> >> >> >
> >> > > > > > > >> > >> >> >
> >> > > > > > > >> > >> >
> >> > > > > > > >> > >
> >> > > > > > > >>
> >> > > > > > > >
> >> > > > > > >
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
 		 	   		  
Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message