streams-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Geer <ch...@cxtsoftware.com>
Subject Re: [DISCUSS] Switching Streams from Camel deployment to .war deployment
Date Fri, 01 Nov 2013 16:48:01 GMT
On Fri, Nov 1, 2013 at 6:56 AM, Danny Sullivan <dsullivan7@hotmail.com>wrote:

> Perhaps it would be best for me to take a step back and ask what problem
> are we trying to solve by using Camel? My understanding is that it is to
> allow distribution of Streams to multiple servers. It would be helpful to
> me to see another open source application that has this deployment
> structure (it doesn't have to use Camel, just an application that has
> multiple jars communicating with one another). Does anyone have any
> suggestions of software for me to look at?
>

Danny, I wish I could just show you my system we are building. Honestly I
think you are looking at this the wrong way. You are looking at it from a
software development perspective (jars, protocols) and not from a
business/system perspective (messages, components). So let me use an
example.

Let's say I have a business that takes order for widgets. I want the order
taking to be streamlined for the user. To make that happen my order entry
system does some basic checks when the user is placing the order and if
those pass it saves the order to the database. To kick off the other
processing, whenever I save an order to the database I sent a message to a
Topic (pub/sub) saying either an order was created or an order was updated.
Now at this point there is a lot more work I have to do to get the order
shipped to the customer but it's an async action to the order taking. At
this point the order has been placed and the customer can go about their
business.

I also have a component I wrote that listens to the Topic for "Order
Created" messages. It takes each message, retrieves the order does some
inventory processing to get the order ready to ship. At this point the
order is complete and shipped to the customer.

Now, in this simple example you could argue that the order entry system
could just call the order processing component directly but that would
preclude some extensibility in the future.

A few months down the road I get a requirement to be able to notify the
sales team whenever an order is placed by a certain set of customers. In a
traditional model I'd have to modify the order entry component and put in
logic to handle that. In this architecture all I have to do is build a
different component that listens to that same Topic for order by certain
customers. When it gets a notification it can then do the logic to send the
message. The beauty of this is the order entry system doesn't' have to be
modified and there is no risk of inserting a bug into that core process.

So you ask, where does camel come in here. It could be used in lots of
places but the natural fit is for the shipment processing component to be a
camel route that listens to the Topic and calls a java object when messages
come in. Also for the notification components, the whole thing could
probably be a camel route (0 Java code).

This is a pretty simple example and there are lots of other benefits I
didn't talk about like redundancy and deployment flexibility.

Does that help?

>
> > Date: Thu, 31 Oct 2013 20:23:26 -0400
> > Subject: Re: [DISCUSS] Switching Streams from Camel deployment to .war
> deployment
> > From: jletourneau80@gmail.com
> > To: dev@streams.incubator.apache.org
> >
> > So Danny in case it was lost in the ensuing transport discussion,
> > Camel abstracts the transport, so you don't have to care if its http
> > or whatever, only about the message format.  This is helpful if you
> > care about plugging in components that DO care about the transport of
> > course.  To answer your other questions, the downside to having the
> > components all work via http (and not using messaging/Camel) is that
> > everything immediately becomes point to point, which isn't a
> > performance issue in its own right but limits scaling to more than one
> > component being at the receiving end/fulfillment for instance as we've
> > been discussing as being optimal.
> >
> > On Thu, Oct 31, 2013 at 5:38 PM, Matt Franklin <m.ben.franklin@gmail.com>
> wrote:
> > > On Thu, Oct 31, 2013 at 5:34 PM, Chris Geer <chris@cxtsoftware.com>
> wrote:
> > >
> > >> On Thu, Oct 31, 2013 at 2:32 PM, Matt Franklin <
> m.ben.franklin@gmail.com
> > >> >wrote:
> > >>
> > >> > On Thu, Oct 31, 2013 at 5:28 PM, Chris Geer <chris@cxtsoftware.com>
> > >> wrote:
> > >> >
> > >> > > On Thu, Oct 31, 2013 at 2:23 PM, Matt Franklin <
> > >> m.ben.franklin@gmail.com
> > >> > > >wrote:
> > >> > >
> > >> > > > On Thu, Oct 31, 2013 at 5:04 PM, Danny Sullivan <
> > >> > dsullivan7@hotmail.com
> > >> > > > >wrote:
> > >> > > >
> > >> > > > > I'm not quite following this, so I apologize. What I'm trying
> to do
> > >> > is
> > >> > > > > programmatically make a request to a jar running on a
> separate jvm
> > >> > and
> > >> > > > get
> > >> > > > > the response from that call all within the same method.
> Similar to
> > >> > this
> > >> > > > > http request:
> > >> > > > >
> > >> > > > > HttpGet httpget = new HttpGet();
> > >> > > > >
> > >> > > > > httpget.setURI(new URI("www.streams-persistence.com"));
> > >> > > > >
> > >> > > > > CloseableHttpResponse response = httpclient.execute(httpget);
> > >> > > > >
> > >> > > > > //do stuff with the response...
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > I imagine that this would translate to something in Camel
> similar
> > >> to
> > >> > > > this:
> > >> > > > >
> > >> > > > >
> > >> > > > > <route>
> > >> > > > >
> > >> > > > > <from uri="bean:subscriberService?method=getAllSubscribers"/>
> > >> > > > >
> > >> > > > > <inOut
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
> uri="addressToStreamsPersistenceJar?method=selectAllSubscribersFromDatabase"/>
> > >> > > > >
> > >> > > > > </route>
> > >> > > > >
> > >> > > >
> > >> > > > Camel uses AMQP as a messaging system so that you don't have to
> > >> create
> > >> > > HTTP
> > >> > > > requests between sources, though it is possible to do so.  In
> Storm,
> > >> we
> > >> > > > usually try to use a buffer system like Kafka to do the same.
> > >> > > >
> > >> > >
> > >> > > Camel can use AMQP through ActiveMQ.
> > >> > >
> > >> > >
> > >> > Can being the operative word.  I haven't done much with Camel.  What
> > >> other
> > >> > transports are available.
> > >> >
> > >> > OpenWire (native ActiveMQ)
> > >> AMQP
> > >> STOMP
> > >> HTTP (can be a server or client)
> > >> Mail
> > >> Amazon SQS
> > >> XMPP
> > >> ....hundred or so more
> > >>
> > >> http://camel.apache.org/components.html
> > >
> > >
> > >
> > > Nice.  I was expecting RTFM, so thanks for catering to my laziness.
> > >
> > >
> > >>
> > >>
> > >> >
> > >> > > >
> > >> > > >
> > >> > > > >
> > >> > > > >
> > >> > > > > But it is unclear what the actual implementation would be.
> > >> > > > >
> > >> > > > >
> > >> > > > > This actually brings me to another suggestion. Would there be
> a big
> > >> > > > > performance impact to have communication between the software
> > >> > > components
> > >> > > > > occur between http? Say the 5 software components I outlined
> > >> earlier
> > >> > > were
> > >> > > > > packaged as 5 separate wars. These wars could communicate
> with each
> > >> > > other
> > >> > > > > via get a post requests. This sounds unconventional offhand
> so I'd
> > >> > like
> > >> > > > to
> > >> > > > > hear some thoughts on it.
> > >> > > > >
> > >> > > >
> > >> > > > It is most certainly possible, but IMO probably not the best
> option
> > >> for
> > >> > > > success.  Protocols like PubSubHubbub use HTTP for a message
> > >> transport.
> > >> > > >
> > >> > > >
> > >> > > > > -Danny
> > >> > > > >
> > >> > > > > > Date: Thu, 31 Oct 2013 13:28:41 -0700
> > >> > > > > > Subject: Re: [DISCUSS] Switching Streams from Camel
> deployment to
> > >> > > .war
> > >> > > > > deployment
> > >> > > > > > From: chris@cxtsoftware.com
> > >> > > > > > To: dev@streams.incubator.apache.org
> > >> > > > > >
> > >> > > > > > Or Content Enricher [2]
> > >> > > > > >
> > >> > > > > > [2] http://camel.apache.org/content-enricher.html
> > >> > > > > >
> > >> > > > > >
> > >> > > > > > On Thu, Oct 31, 2013 at 12:37 PM, Jason Letourneau
> > >> > > > > > <jletourneau80@gmail.com>wrote:
> > >> > > > > >
> > >> > > > > > > check out the link here[1]
> > >> > > > > > >
> > >> > > > > > >
> > >> > > > > > > [1]http://camel.apache.org/request-reply.html
> > >> > > > > > >
> > >> > > > > > > On Thu, Oct 31, 2013 at 2:32 PM, Danny Sullivan <
> > >> > > > > dsullivan7@hotmail.com>
> > >> > > > > > > wrote:
> > >> > > > > > > > I have a quick Camel question that I arrived at in the
> > >> > > > > implementation of
> > >> > > > > > > these new components:
> > >> > > > > > > >
> > >> > > > > > > > Lets say I have a method in streams-activity.jar that
> needs
> > >> all
> > >> > > > > > > subscribers in the database. This would require a call to
> the
> > >> > > > > > > streams-persistence.jar. So far, I've seen camel used
> mostly
> > >> for
> > >> > > > > passing
> > >> > > > > > > data through the application, but not for making a single
> > >> > > > > request-reponse
> > >> > > > > > > from within a method. How can I use Camel to get a list
> of all
> > >> > > > > subscribers
> > >> > > > > > > in the streams-persistence.jar from the
> streams-activity.jar?
> > >> > > > > > > >
> > >> > > > > > > >> From: dsullivan7@hotmail.com
> > >> > > > > > > >> To: dev@streams.incubator.apache.org
> > >> > > > > > > >> Subject: RE: [DISCUSS] Switching Streams from Camel
> > >> deployment
> > >> > > to
> > >> > > > > .war
> > >> > > > > > > deployment
> > >> > > > > > > >> Date: Thu, 31 Oct 2013 08:40:44 -0400
> > >> > > > > > > >>
> > >> > > > > > > >> Excellent, I'll write up something as a proof of
> concept and
> > >> > we
> > >> > > > can
> > >> > > > > > > discuss further to make sure everything is vanilla.
> > >> > > > > > > >>
> > >> > > > > > > >> > Date: Wed, 30 Oct 2013 17:01:28 -0400
> > >> > > > > > > >> > Subject: Re: [DISCUSS] Switching Streams from Camel
> > >> > deployment
> > >> > > > to
> > >> > > > > > > .war deployment
> > >> > > > > > > >> > From: jletourneau80@gmail.com
> > >> > > > > > > >> > To: dev@streams.incubator.apache.org
> > >> > > > > > > >> >
> > >> > > > > > > >> > That sounds pretty promising to me.
> > >> > > > > > > >> >
> > >> > > > > > > >> > On Wed, Oct 30, 2013 at 2:38 PM, Danny Sullivan <
> > >> > > > > > > dsullivan7@hotmail.com> wrote:
> > >> > > > > > > >> > > Thanks for the feedback. You have an interesting
> point
> > >> > about
> > >> > > > the
> > >> > > > > > > url linking to a separate processing space. Let me tie my
> > >> answer
> > >> > > into
> > >> > > > > your
> > >> > > > > > > last question about "advocating for the simplicity at
> > >> > registration
> > >> > > to
> > >> > > > > give
> > >> > > > > > > up flexibility at registration, but retaining the inner
> "guts"
> > >> of
> > >> > > > > > > EIP/messaging". Consider a new architecture:
> > >> > > > > > > >> > >
> > >> > > > > > > >> > > streams-web.war: single entry point to
> application, but
> > >> > > > > functions
> > >> > > > > > > ONLY as an entry point. From here Camel routes the
> incoming
> > >> > > requests
> > >> > > > > to 4
> > >> > > > > > > separate jarssubscriber-registration.jar: subscriber
> > >> registration
> > >> > > > > > > publisher-registration.jar: publisher
> registrationactivity.jar:
> > >> > > > returns
> > >> > > > > > > activity (also contains subscriber warehouse and storm
> activity
> > >> > > > > > > aggregator)publish.jar: publishes
> > >> activitystreams-cassandra.jar:
> > >> > > the
> > >> > > > > above
> > >> > > > > > > 4 jars would all have a hook into this jar which would
> function
> > >> > as
> > >> > > a
> > >> > > > > hook
> > >> > > > > > > onto the database. Each jar would have camel route output
> to
> > >> this
> > >> > > > jar.
> > >> > > > > > > >> > >
> > >> > > > > > > >> > > In this implementation, Camel would no longer be
> the
> > >> entry
> > >> > > and
> > >> > > > > exit
> > >> > > > > > > point of a client to the application, but would handle the
> > >> > > > > communication
> > >> > > > > > > between components. The flow of activity through the
> > >> application
> > >> > > > would
> > >> > > > > be
> > >> > > > > > > method based in each jar. This would allow deployment on
> up to
> > >> 6
> > >> > > > > different
> > >> > > > > > > process spaces. However, this does not address that there
> is a
> > >> > > single
> > >> > > > > > > server entry point, but I'm not sure if it was a concern
> in the
> > >> > > first
> > >> > > > > place.
> > >> > > > > > > >> > >
> > >> > > > > > > >> > > My argument, at its basis, is that we should move
> away
> > >> > from
> > >> > > > > using
> > >> > > > > > > Camel as the entry point to the application. I would be
> happy
> > >> to
> > >> > > > > maintain
> > >> > > > > > > messaging between components.
> > >> > > > > > > >> > >
> > >> > > > > > > >> > >> Date: Wed, 30 Oct 2013 12:32:55 -0400
> > >> > > > > > > >> > >> Subject: Re: [DISCUSS] Switching Streams from
> Camel
> > >> > > > deployment
> > >> > > > > to
> > >> > > > > > > .war deployment
> > >> > > > > > > >> > >> From: jletourneau80@gmail.com
> > >> > > > > > > >> > >> To: dev@streams.incubator.apache.org
> > >> > > > > > > >> > >>
> > >> > > > > > > >> > >> An interesting use case that I am holding onto is
> the
> > >> > > ability
> > >> > > > > for
> > >> > > > > > > >> > >> publishers to register via a single URL
> (registration
> > >> > > > > endpoint),
> > >> > > > > > > but
> > >> > > > > > > >> > >> be sent a URL back to post to a different process
> space
> > >> > for
> > >> > > > > actual
> > >> > > > > > > >> > >> publishing.  The same is true on the subscriber
> front.
> > >> > > > >  Currently,
> > >> > > > > > > the
> > >> > > > > > > >> > >> Camel/EIP infrastructure abstracts this because
> > >> different
> > >> > > > > > > components
> > >> > > > > > > >> > >> deployed in different process spaces handling the
> route
> > >> > > > > creation
> > >> > > > > > > can
> > >> > > > > > > >> > >> just be bolted onto a running Streams instance
> without
> > >> > new
> > >> > > > > > > subs/pubs
> > >> > > > > > > >> > >> behaving any differently than existing.  This
> seems to
> > >> > be a
> > >> > > > > > > >> > >> potentially critical scaling point.  Is there a
> way to
> > >> do
> > >> > > > this
> > >> > > > > with
> > >> > > > > > > >> > >> the Spring solution?
> > >> > > > > > > >> > >>
> > >> > > > > > > >> > >> The persistence point is a good one, though I
> would
> > >> > > classify
> > >> > > > > that
> > >> > > > > > > as
> > >> > > > > > > >> > >> "not implemented" vs "not possible" (not that you
> > >> were).
> > >> > > > > > > >> > >>
> > >> > > > > > > >> > >> I'm not married to Camel, I just like the EIP
> approach
> > >> to
> > >> > > > > building
> > >> > > > > > > >> > >> something that is ultimately a messaging system.
>  There
> > >> > are
> > >> > > > > known
> > >> > > > > > > >> > >> patterns that solve at least a subset of the
> problems
> > >> > > Streams
> > >> > > > > is
> > >> > > > > > > >> > >> trying to solve and implementations that can
> handle the
> > >> > > load
> > >> > > > > and
> > >> > > > > > > I'll
> > >> > > > > > > >> > >> reiterate flexibility == complexity almost always.
> > >> > > > > > > >> > >>
> > >> > > > > > > >> > >> It comes right back to the central question: Do
> you
> > >> want
> > >> > > > > > > flexibility
> > >> > > > > > > >> > >> or simplicity?  It doesn't have to be black and
> white
> > >> > > either
> > >> > > > I
> > >> > > > > > > don't
> > >> > > > > > > >> > >> think...
> > >> > > > > > > >> > >>
> > >> > > > > > > >> > >> More pointedly: Where should we give up
> flexibility for
> > >> > > > > > > simplicity?  I
> > >> > > > > > > >> > >> read that Danny is advocating for the simplicity
> at
> > >> > > > > registration to
> > >> > > > > > > >> > >> give up flexibility at registration, but
> retaining the
> > >> > > inner
> > >> > > > > > > "guts" of
> > >> > > > > > > >> > >> EIP/messaging?  Thoughts?
> > >> > > > > > > >> > >>
> > >> > > > > > > >> > >> On Wed, Oct 30, 2013 at 11:37 AM, Danny Sullivan <
> > >> > > > > > > dsullivan7@hotmail.com> wrote:
> > >> > > > > > > >> > >> > My argument is not for the IoC pattern as that
> can be
> > >> > > (and
> > >> > > > > has
> > >> > > > > > > been) implemented alongside Camel. My main argument is
> that the
> > >> > > > syntax
> > >> > > > > at
> > >> > > > > > > the entry point is not only familiar but much simpler.
> This
> > >> > > wouldn't
> > >> > > > > be a
> > >> > > > > > > very strong argument if the Camel implementation wasn't
> much
> > >> more
> > >> > > > > > > complicated but I feel that it is the case. Also, looking
> > >> toward
> > >> > > the
> > >> > > > > > > future, if the server is restarted, in-routes are lost in
> > >> Camel.
> > >> > > The
> > >> > > > > way to
> > >> > > > > > > curb this is to persist the dynamic routes that Camel
> creates,
> > >> > and
> > >> > > > > then on
> > >> > > > > > > start up pull every one of these routes and recreate a
> dynamic
> > >> > > route
> > >> > > > > for
> > >> > > > > > > each one. Not only is this much easier to implement using
> the
> > >> > > Spring
> > >> > > > > web
> > >> > > > > > > implementation, but it already has been implemented and
> you can
> > >> > try
> > >> > > > it
> > >> > > > > by
> > >> > > > > > > checking out the webservice branch, registering a
> subscriber,
> > >> > > > > restarting
> > >> > > > > > > tomcat, and using the same url you had before. This will
> allow
> > >> > > > > subscribers
> > >> > > > > > > to hang on to their urls once they register. (the same is
> true
> > >> > for
> > >> > > > > > > publishers: you can post via the same url after restarting
> > >> > tomcat)
> > >> > > > > > > >> > >> >
> > >> > > > > > > >> > >> >> Date: Wed, 30 Oct 2013 11:00:21 -0400
> > >> > > > > > > >> > >> >> Subject: Re: [DISCUSS] Switching Streams from
> Camel
> > >> > > > > deployment
> > >> > > > > > > to .war deployment
> > >> > > > > > > >> > >> >> From: jletourneau80@gmail.com
> > >> > > > > > > >> > >> >> To: dev@streams.incubator.apache.org
> > >> > > > > > > >> > >> >>
> > >> > > > > > > >> > >> >> To be fair, while the current implementation is
> > >> > heavily
> > >> > > > > > > camel-based,
> > >> > > > > > > >> > >> >> all of the interfaces related to Streams
> > >> functionality
> > >> > > are
> > >> > > > > not.
> > >> > > > > > >  The
> > >> > > > > > > >> > >> >> current model maps to what Matt has outlined
> in my
> > >> > > > opinion,
> > >> > > > > > > though
> > >> > > > > > > >> > >> >> packing names etc. probably don't follow that
> exact
> > >> > > > pattern.
> > >> > > > > > > >> > >> >>
> > >> > > > > > > >> > >> >> With regards to the complexity and different
> > >> > components
> > >> > > in
> > >> > > > > the
> > >> > > > > > > >> > >> >> registration process, this was a cut at the
> > >> > abstraction
> > >> > > > > based
> > >> > > > > > > on the
> > >> > > > > > > >> > >> >> assumption that different implementations may
> be
> > >> > plugged
> > >> > > > in
> > >> > > > > and
> > >> > > > > > > in
> > >> > > > > > > >> > >> >> fact may live on different processor space
> (ie. a
> > >> > > polling
> > >> > > > > > > publisher vs
> > >> > > > > > > >> > >> >> a push publisher may be instantiated on
> different
> > >> > > servers
> > >> > > > > but
> > >> > > > > > > the
> > >> > > > > > > >> > >> >> registration URL is staticly defined).
> > >> > > > > > > >> > >> >>
> > >> > > > > > > >> > >> >> Is the main argument I am seeing for Spring
>  the
> > >> > > > > familiarity of
> > >> > > > > > > its
> > >> > > > > > > >> > >> >> IoC pattern implementation and syntax at the
> entry
> > >> > > point?
> > >> > > > > > > >> > >> >>
> > >> > > > > > > >> > >> >> On Wed, Oct 30, 2013 at 10:53 AM, Danny
> Sullivan <
> > >> > > > > > > dsullivan7@hotmail.com> wrote:
> > >> > > > > > > >> > >> >> > Could you clarify whether the same entry
> points
> > >> > would
> > >> > > > > exist
> > >> > > > > > > for the camel implementation of the core (implementing the
> > >> > > "process"
> > >> > > > > > > method/ using a DynammicRouteBuilder) or would the
> webservice
> > >> be
> > >> > > the
> > >> > > > > sole
> > >> > > > > > > entry point to Streams and after it enters would it hand
> it off
> > >> > to
> > >> > > > > Camel?
> > >> > > > > > > And what would be the entry point for the Storm
> implementation?
> > >> > > > > > > >> > >> >> > -Danny
> > >> > > > > > > >> > >> >> >
> > >> > > > > > > >> > >> >> >> Date: Wed, 30 Oct 2013 10:13:04 -0400
> > >> > > > > > > >> > >> >> >> Subject: Re: [DISCUSS] Switching Streams
> from
> > >> Camel
> > >> > > > > > > deployment to .war deployment
> > >> > > > > > > >> > >> >> >> From: m.ben.franklin@gmail.com
> > >> > > > > > > >> > >> >> >> To: dev@streams.incubator.apache.org
> > >> > > > > > > >> > >> >> >>
> > >> > > > > > > >> > >> >> >> On Tue, Oct 29, 2013 at 2:55 PM, Danny
> Sullivan <
> > >> > > > > > > dsullivan7@hotmail.com>wrote:
> > >> > > > > > > >> > >> >> >>
> > >> > > > > > > >> > >> >> >> > My case for switching from OSGi is for
> > >> simplicity
> > >> > > in
> > >> > > > > > > design. To follow the
> > >> > > > > > > >> > >> >> >> > path of an activity through Streams in the
> > >> > > > webservice,
> > >> > > > > > > there is one main
> > >> > > > > > > >> > >> >> >> > things the developer needs to understand:
> > >> > > > > > > >> > >> >> >> > The @RequestMapping annotation specifies
> the
> > >> HTTP
> > >> > > > entry
> > >> > > > > > > point
> > >> > > > > > > >> > >> >> >> > There are 4 @RequestMapping annotations
> that
> > >> > > > > correspond to
> > >> > > > > > > each of the 4
> > >> > > > > > > >> > >> >> >> > ways a user enters the application:
> > >> registering a
> > >> > > > > > > publisher, registering a
> > >> > > > > > > >> > >> >> >> > subscriber, publishing activity, and
> getting an
> > >> > > > > activity
> > >> > > > > > > stream. Where
> > >> > > > > > > >> > >> >> >> > these are located in the source code can
> be
> > >> found
> > >> > > by
> > >> > > > > > > searching for the
> > >> > > > > > > >> > >> >> >> > paths specified in the documentation
> (search
> > >> for
> > >> > > > > > > "/publisherRegister",
> > >> > > > > > > >> > >> >> >> > "/subscriberRegister", "/publishActivity",
> > >> > > > > "/getActivity"
> > >> > > > > > > which will all
> > >> > > > > > > >> > >> >> >> > lead you to StreamsWebController.java).
> From
> > >> the
> > >> > > > > methods
> > >> > > > > > > that process
> > >> > > > > > > >> > >> >> >> > requests, the flow through the
> application is
> > >> > > through
> > >> > > > > > > methods which can be
> > >> > > > > > > >> > >> >> >> > understood by most Java programmers.
> > >> > > > > > > >> > >> >> >> > The flow of activities through the current
> > >> trunk
> > >> > > > > branch is
> > >> > > > > > > understood as
> > >> > > > > > > >> > >> >> >> > follows:
> > >> > > > > > > >> > >> >> >> > The string "/publisher/register" (the
> entry
> > >> point
> > >> > > to
> > >> > > > > > > register a publisher
> > >> > > > > > > >> > >> >> >> > specified in the documentation) is the
> value of
> > >> > the
> > >> > > > > > > >> > >> >> >> > consumer.registrationEndpoint property
> defined
> > >> in
> > >> > > > > > > streams.propertiesThe
> > >> > > > > > > >> > >> >> >> > camelContext.xml specifies an endpoint
> with the
> > >> > id
> > >> > > > > > > >> > >> >> >> > consumerRegistrationEndpoint to have a uri
> > >> equal
> > >> > to
> > >> > > > the
> > >> > > > > > > propertyThe
> > >> > > > > > > >> > >> >> >> > consumerRegistrationEndpoint routes from
> uri
> > >> > > > > > > direct:publisher register with
> > >> > > > > > > >> > >> >> >> > the bean activityRegistrationProcessor
> nested
> > >> in
> > >> > > > > between
> > >> > > > > > > the routeThe
> > >> > > > > > > >> > >> >> >> > streams-eip-applicationContext contains a
> bean
> > >> > with
> > >> > > > > the id
> > >> > > > > > > >> > >> >> >> > activityRegistrationProcessor created for
> the
> > >> > class
> > >> > > > > > > >> > >> >> >> > ActivityPublisherRegistrationProcessorThe
> > >> > exchange
> > >> > > > will
> > >> > > > > > > enter the "process"
> > >> > > > > > > >> > >> >> >> > method of the class
> > >> > > > > > > ActivityPublisherRegisitrationProcessor and that this
> > >> > > > > > > >> > >> >> >> > is because this class implements the
> > >> "Processor"
> > >> > > > > interface
> > >> > > > > > > provided by
> > >> > > > > > > >> > >> >> >> > CamelThe direct:add-publisher-route takes
> the
> > >> > > > exchange
> > >> > > > > > > output from the
> > >> > > > > > > >> > >> >> >> > "process" method and routes it to the
> > >> > > > > > > activityRegistrationProcessor
> > >> > > > > > > >> > >> >> >> > "register" method. The bean
> > >> > > > > activityRegistrationProcessor
> > >> > > > > > > is defined in the
> > >> > > > > > > >> > >> >> >> > streams-eip-osgi-component-import.xmlThe
> output
> > >> > > from
> > >> > > > > this
> > >> > > > > > > method is then
> > >> > > > > > > >> > >> >> >> > sent to the "createNewRouteForConsumer"
> method
> > >> of
> > >> > > > > > > activityConsumerRouter.
> > >> > > > > > > >> > >> >> >> > This method creates a new route for the
> newly
> > >> > > > > registered
> > >> > > > > > > publisher using
> > >> > > > > > > >> > >> >> >> > the private static final class
> > >> > > > > DynamicConsumerRouteBuilder
> > >> > > > > > > which is
> > >> > > > > > > >> > >> >> >> > required to extend RouteBuilder which is
> > >> provided
> > >> > > by
> > >> > > > > > > Camel. This
> > >> > > > > > > >> > >> >> >> > DynamicConsumerRouteBuilder contains
> several
> > >> > > methods:
> > >> > > > > > > >> > >> >> >> > getConsumerReceiveMethod() (which
> corresponds
> > >> to
> > >> > > > @Value
> > >> > > > > > > >> > >> >> >> > ${consumer.receiveMethod} which
> corresponds to
> > >> > > > > "receive"),
> > >> > > > > > > >> > >> >> >> > getConsumerSplitMethod() (@Value
> > >> > > > > > > ${consumer.splitMethod},"split"), and
> > >> > > > > > > >> > >> >> >> > getConsumerActivityQUri() (@Value
> > >> > > > > ${consumerActivityQUri},
> > >> > > > > > > >> > >> >> >> > "direct:activityQ"). This is different
> than the
> > >> > > > > > > camelContext.xml in that
> > >> > > > > > > >> > >> >> >> > the route is being created
> programmatically.
> > >> What
> > >> > > > this
> > >> > > > > is
> > >> > > > > > > doing is routing
> > >> > > > > > > >> > >> >> >> > input from the inroute url (which Camel
> does
> > >> > > > > automatically
> > >> > > > > > > through the
> > >> > > > > > > >> > >> >> >> > configure method which is required to be
> > >> > > overridden),
> > >> > > > > to
> > >> > > > > > > the "receive"
> > >> > > > > > > >> > >> >> >> > method of ActivityConsumer, then to the
> "split"
> > >> > > > method
> > >> > > > > of
> > >> > > > > > > ActivityConsumer,
> > >> > > > > > > >> > >> >> >> > and then to the "direct:activityQ" which
> if you
> > >> > > look
> > >> > > > > back
> > >> > > > > > > in the
> > >> > > > > > > >> > >> >> >> > camelContext.xml routes to the
> > >> > > > > "activemq:queue:activities"
> > >> > > > > > > which then
> > >> > > > > > > >> > >> >> >> > routes to "receiveExchange"
> > >> > > > > > > >> > >> >> >> > This is the process to register a
> publisher.
> > >> The
> > >> > > > > process
> > >> > > > > > > for registering a
> > >> > > > > > > >> > >> >> >> > subscriber is relatively the same though
> it
> > >> > > involves
> > >> > > > > > > separate classes with
> > >> > > > > > > >> > >> >> >> > their own private static final
> RouteBuilder
> > >> > class.
> > >> > > > > From my
> > >> > > > > > > perspective, the
> > >> > > > > > > >> > >> >> >> > two most difficult things with setting
> this
> > >> > project
> > >> > > > up
> > >> > > > > > > were understanding
> > >> > > > > > > >> > >> >> >> > that the "process" method of the class
> that
> > >> > > > implements
> > >> > > > > > > "Processor" is the
> > >> > > > > > > >> > >> >> >> > entry point and the
> DynamicConsumerRouteBuilder
> > >> > > > creates
> > >> > > > > > > the second entry
> > >> > > > > > > >> > >> >> >> > point (The 5th and last points). This
> made the
> > >> > > > project
> > >> > > > > > > very, VERY hard to
> > >> > > > > > > >> > >> >> >> > understand.
> > >> > > > > > > >> > >> >> >> > In addition to simplicity of design, the
> mvn
> > >> > clean
> > >> > > > > install
> > >> > > > > > > of the web
> > >> > > > > > > >> > >> >> >> > service project is much faster and small
> scale
> > >> > > > activity
> > >> > > > > > > publishing is also
> > >> > > > > > > >> > >> >> >> > faster (see my email about load testing).
> These
> > >> > are
> > >> > > > > minor
> > >> > > > > > > points though as
> > >> > > > > > > >> > >> >> >> > compilation has no effect on deployment.
> OSGi
> > >> > does
> > >> > > > add
> > >> > > > > the
> > >> > > > > > > benefit of
> > >> > > > > > > >> > >> >> >> > modularized programming which is valuable,
> > >> > though I
> > >> > > > > think
> > >> > > > > > > the added
> > >> > > > > > > >> > >> >> >> > complexity of Camel merits moving the
> project
> > >> > away
> > >> > > > from
> > >> > > > > > > this paradigm.
> > >> > > > > > > >> > >> >> >> >
> > >> > > > > > > >> > >> >> >>
> > >> > > > > > > >> > >> >> >> I agree that the project is pretty
> difficult to
> > >> > > > > understand
> > >> > > > > > > ATM.  I think
> > >> > > > > > > >> > >> >> >> what we need to do is think about what the
> > >> > > > > responsibilities
> > >> > > > > > > of the code are
> > >> > > > > > > >> > >> >> >> and allow for different implementations
> that are
> > >> > not
> > >> > > so
> > >> > > > > > > tightly coupled as
> > >> > > > > > > >> > >> >> >> they are now.  For instance, having worked
> with
> > >> > Storm
> > >> > > > to
> > >> > > > > > > ingest millions of
> > >> > > > > > > >> > >> >> >> activities a day, I personally would like
> to see
> > >> > > > streams
> > >> > > > > be
> > >> > > > > > > responsible for
> > >> > > > > > > >> > >> >> >> defining an over-arching orchestration
> model that
> > >> > can
> > >> > > > be
> > >> > > > > > > implemented within
> > >> > > > > > > >> > >> >> >> a single war or on top of a distributed
> system.
> > >> >  This
> > >> > > > > would
> > >> > > > > > > look something
> > >> > > > > > > >> > >> >> >> like the follows:
> > >> > > > > > > >> > >> >> >>
> > >> > > > > > > >> > >> >> >> |__ Streams-Core  (Base classes, interfaces,
> > >> > > utilities,
> > >> > > > > > > extensions, etc)
> > >> > > > > > > >> > >> >> >> |
> > >> > > > > > > >> > >> >> >> |__ Streams-Storm (Storm implementation of
> the
> > >> > core)
> > >> > > > > > > >> > >> >> >> |
> > >> > > > > > > >> > >> >> >> |__ Streams-Camel (Camel implementation of
> the
> > >> > core)
> > >> > > > > > > >> > >> >> >> |
> > >> > > > > > > >> > >> >> >> |__ Streams-WS (Web service implementation)
> > >> > > > > > > >> > >> >> >>
> > >> > > > > > > >> > >> >> >>
> > >> > > > > > > >> > >> >> >> > Danny
> > >> > > > > > > >> > >> >> >> >
> > >> > > > > > > >> > >> >> >> >
> > >> > > > > > > >> > >> >> >> > > From: dsullivan7@hotmail.com
> > >> > > > > > > >> > >> >> >> > > To: dev@streams.incubator.apache.org
> > >> > > > > > > >> > >> >> >> > > Subject: [DISCUSS] Switching Streams
> from
> > >> Camel
> > >> > > > > > > deployment to .war
> > >> > > > > > > >> > >> >> >> > deployment
> > >> > > > > > > >> > >> >> >> > > Date: Tue, 29 Oct 2013 11:07:39 -0400
> > >> > > > > > > >> > >> >> >> > >
> > >> > > > > > > >> > >> >> >> > > The discussion thread for switching
> Streams
> > >> > from
> > >> > > > > > > Camel/osgi/Servicemix
> > >> > > > > > > >> > >> >> >> > to a single .war deployment
> > >> > > > > > > >> > >> >> >> >
> > >> > > > > > > >> > >> >> >> >
> > >> > > > > > > >> > >> >> >
> > >> > > > > > > >> > >> >
> > >> > > > > > > >> > >
> > >> > > > > > > >>
> > >> > > > > > > >
> > >> > > > > > >
> > >> > > > >
> > >> > > > >
> > >> > > >
> > >> > >
> > >> >
> > >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message