Return-Path: 
 <dev-return-475-apmail-streams-dev-archive=streams.apache.org@streams.incubator.apache.org>
X-Original-To: apmail-streams-dev-archive@minotaur.apache.org
Delivered-To: apmail-streams-dev-archive@minotaur.apache.org
Received: from mail.apache.org (hermes.apache.org [140.211.11.3])
	by minotaur.apache.org (Postfix) with SMTP id 2C60EC914
	for <apmail-streams-dev-archive@minotaur.apache.org>;
 Fri,  1 Nov 2013 13:57:21 +0000 (UTC)
Received: (qmail 95157 invoked by uid 500); 1 Nov 2013 13:57:19 -0000
Delivered-To: apmail-streams-dev-archive@streams.apache.org
Received: (qmail 95104 invoked by uid 500); 1 Nov 2013 13:57:15 -0000
Mailing-List: contact dev-help@streams.incubator.apache.org; run by ezmlm
Precedence: bulk
List-Help: <mailto:dev-help@streams.incubator.apache.org>
List-Unsubscribe: <mailto:dev-unsubscribe@streams.incubator.apache.org>
List-Post: <mailto:dev@streams.incubator.apache.org>
List-Id: <dev.streams.incubator.apache.org>
Reply-To: dev@streams.incubator.apache.org
Delivered-To: mailing list dev@streams.incubator.apache.org
Received: (qmail 95096 invoked by uid 99); 1 Nov 2013 13:57:13 -0000
Received: from athena.apache.org (HELO athena.apache.org) (140.211.11.136)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Nov 2013 13:57:13 +0000
X-ASF-Spam-Status: No, hits=2.4 required=5.0
	tests=FREEMAIL_ENVFROM_END_DIGIT,HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS
X-Spam-Check-By: apache.org
Received-SPF: pass (athena.apache.org: domain of dsullivan7@hotmail.com
 designates 65.54.51.88 as permitted sender)
Received: from [65.54.51.88] (HELO snt0-omc3-s51.snt0.hotmail.com)
 (65.54.51.88)
    by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 01 Nov 2013 13:57:08 +0000
Received: from SNT149-W33 ([65.55.90.136]) by snt0-omc3-s51.snt0.hotmail.com
 with Microsoft SMTPSVC(6.0.3790.4675);
	 Fri, 1 Nov 2013 06:56:47 -0700
X-TMN: [GJQnf678YTLVjmk4PtUuhwnT+ge/c+Af]
X-Originating-Email: [dsullivan7@hotmail.com]
Message-ID: <SNT149-W336BB48418DA72C693B9D18AF50@phx.gbl>
Content-Type: multipart/alternative;
	boundary="_96523367-9983-4688-8ef4-e757902a84aa_"
From: Danny Sullivan <dsullivan7@hotmail.com>
To: "dev@streams.incubator.apache.org" <dev@streams.incubator.apache.org>
Subject: RE: [DISCUSS] Switching Streams from Camel deployment to .war
 deployment
Date: Fri, 1 Nov 2013 09:56:46 -0400
Importance: Normal
In-Reply-To: 
 <CAL8syOY-i8+ur33XOcxk=iHxUkR4RU=FtGRSJCNOed1r=FY8cA@mail.gmail.com>
References: 
 <SNT149-W9255D3543C9374C1010DCB8A090@phx.gbl>,<SNT149-W17DBFEA90CE093DF3F90F88A090@phx.gbl>,<CAENpzYWdGg4xK2Ey+CeFqV2jC5DZWyM8ocRX03Yn7o+p9FnoEg@mail.gmail.com>,<SNT149-W478477E9CE78116FB1F0188A0A0@phx.gbl>,<CAL8syObZtmfDBtznouz5k+bEgcG55=W9k8KvKtQBQVNHzNsTBA@mail.gmail.com>,<SNT149-W4688D1837ED8A61183B6278A0A0@phx.gbl>,<CAL8syOadWnYXMOc3X6ooU-c7Wanjo98YiNHaOU2PC9S5+Bou8Q@mail.gmail.com>,<SNT149-W3466D775B64424835F79A38A0A0@phx.gbl>,<CAL8syOYRWHdQvHo_mysSe01vdNAOpvypTe9dBZycJ4J3w=1x+Q@mail.gmail.com>,<SNT149-W6947AB6A21214358E253618A0B0@phx.gbl>,<SNT149-W607E0AEEE5380DED910ABE8A0B0@phx.gbl>,<CAL8syOa2JQ0Lc4ztRxDBo5ecVFLD3kBq15wAcDhVmtMNc+UAcQ@mail.gmail.com>,<CAFNO4HhB0QbDOYHsbGTL=rPRnyBzActW4gcy3g3N+TxMJT1n4w@mail.gmail.com>,<SNT149-W51AB5CB1F1F30F2D3098588A0B0@phx.gbl>,<CAENpzYUBp71UGwK8Gw5bHL5o91xJOiq9faFby+Ww2DUO3jcYrg@mail.gmail.com>,<CAFNO4HgnCU2XFHe-03fNo15XnPw0dWr5OBnK26cpjfpq-kZT_g@mail.gmail.com>,<CAENpzYXjrutYHsxP+K-GtjO++vhyKYM9+Gay+T+L2PPQ34WfTA@mail.gmail.com>,<CAFNO4H
 i6DV6teLBnWPFhh=ex+6r-CFLizSqQOVa6BSJkr8yDeA@mail.gmail.com>,<CAENpzYU+zi4XWVW4c4p+F28_skQC-etknXXhWY0W24L=-f+fCA@mail.gmail.com>,<CAL8syOY-i8+ur33XOcxk=iHxUkR4RU=FtGRSJCNOed1r=FY8cA@mail.gmail.com>
MIME-Version: 1.0
X-OriginalArrivalTime: 01 Nov 2013 13:56:47.0018 (UTC)
 FILETIME=[3476F8A0:01CED70A]
X-Virus-Checked: Checked by ClamAV on apache.org

--_96523367-9983-4688-8ef4-e757902a84aa_
Content-Type: text/plain; charset="iso-8859-1"
Content-Transfer-Encoding: quoted-printable

Perhaps it would be best for me to take a step back and ask what problem ar=
e we trying to solve by using Camel? My understanding is that it is to allo=
w distribution of Streams to multiple servers. It would be helpful to me to=
 see another open source application that has this deployment structure (it=
 doesn't have to use Camel=2C just an application that has multiple jars co=
mmunicating with one another). Does anyone have any suggestions of software=
 for me to look at?

> Date: Thu=2C 31 Oct 2013 20:23:26 -0400
> Subject: Re: [DISCUSS] Switching Streams from Camel deployment to .war de=
ployment
> From: jletourneau80@gmail.com
> To: dev@streams.incubator.apache.org
>=20
> So Danny in case it was lost in the ensuing transport discussion=2C
> Camel abstracts the transport=2C so you don't have to care if its http
> or whatever=2C only about the message format.  This is helpful if you
> care about plugging in components that DO care about the transport of
> course.  To answer your other questions=2C the downside to having the
> components all work via http (and not using messaging/Camel) is that
> everything immediately becomes point to point=2C which isn't a
> performance issue in its own right but limits scaling to more than one
> component being at the receiving end/fulfillment for instance as we've
> been discussing as being optimal.
>=20
> On Thu=2C Oct 31=2C 2013 at 5:38 PM=2C Matt Franklin <m.ben.franklin@gmai=
l.com> wrote:
> > On Thu=2C Oct 31=2C 2013 at 5:34 PM=2C Chris Geer <chris@cxtsoftware.co=
m> wrote:
> >
> >> On Thu=2C Oct 31=2C 2013 at 2:32 PM=2C Matt Franklin <m.ben.franklin@g=
mail.com
> >> >wrote:
> >>
> >> > On Thu=2C Oct 31=2C 2013 at 5:28 PM=2C Chris Geer <chris@cxtsoftware=
.com>
> >> wrote:
> >> >
> >> > > On Thu=2C Oct 31=2C 2013 at 2:23 PM=2C Matt Franklin <
> >> m.ben.franklin@gmail.com
> >> > > >wrote:
> >> > >
> >> > > > On Thu=2C Oct 31=2C 2013 at 5:04 PM=2C Danny Sullivan <
> >> > dsullivan7@hotmail.com
> >> > > > >wrote:
> >> > > >
> >> > > > > I'm not quite following this=2C so I apologize. What I'm tryin=
g to do
> >> > is
> >> > > > > programmatically make a request to a jar running on a separate=
 jvm
> >> > and
> >> > > > get
> >> > > > > the response from that call all within the same method. Simila=
r to
> >> > this
> >> > > > > http request:
> >> > > > >
> >> > > > > HttpGet httpget =3D new HttpGet()=3B
> >> > > > >
> >> > > > > httpget.setURI(new URI("www.streams-persistence.com"))=3B
> >> > > > >
> >> > > > > CloseableHttpResponse response =3D httpclient.execute(httpget)=
=3B
> >> > > > >
> >> > > > > //do stuff with the response...
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > >
> >> > > > > I imagine that this would translate to something in Camel simi=
lar
> >> to
> >> > > > this:
> >> > > > >
> >> > > > >
> >> > > > > <route>
> >> > > > >
> >> > > > > <from uri=3D"bean:subscriberService?method=3DgetAllSubscribers=
"/>
> >> > > > >
> >> > > > > <inOut
> >> > > > >
> >> > > >
> >> > >
> >> >
> >> uri=3D"addressToStreamsPersistenceJar?method=3DselectAllSubscribersFro=
mDatabase"/>
> >> > > > >
> >> > > > > </route>
> >> > > > >
> >> > > >
> >> > > > Camel uses AMQP as a messaging system so that you don't have to
> >> create
> >> > > HTTP
> >> > > > requests between sources=2C though it is possible to do so.  In =
Storm=2C
> >> we
> >> > > > usually try to use a buffer system like Kafka to do the same.
> >> > > >
> >> > >
> >> > > Camel can use AMQP through ActiveMQ.
> >> > >
> >> > >
> >> > Can being the operative word.  I haven't done much with Camel.  What
> >> other
> >> > transports are available.
> >> >
> >> > OpenWire (native ActiveMQ)
> >> AMQP
> >> STOMP
> >> HTTP (can be a server or client)
> >> Mail
> >> Amazon SQS
> >> XMPP
> >> ....hundred or so more
> >>
> >> http://camel.apache.org/components.html
> >
> >
> >
> > Nice.  I was expecting RTFM=2C so thanks for catering to my laziness.
> >
> >
> >>
> >>
> >> >
> >> > > >
> >> > > >
> >> > > > >
> >> > > > >
> >> > > > > But it is unclear what the actual implementation would be.
> >> > > > >
> >> > > > >
> >> > > > > This actually brings me to another suggestion. Would there be =
a big
> >> > > > > performance impact to have communication between the software
> >> > > components
> >> > > > > occur between http? Say the 5 software components I outlined
> >> earlier
> >> > > were
> >> > > > > packaged as 5 separate wars. These wars could communicate with=
 each
> >> > > other
> >> > > > > via get a post requests. This sounds unconventional offhand so=
 I'd
> >> > like
> >> > > > to
> >> > > > > hear some thoughts on it.
> >> > > > >
> >> > > >
> >> > > > It is most certainly possible=2C but IMO probably not the best o=
ption
> >> for
> >> > > > success.  Protocols like PubSubHubbub use HTTP for a message
> >> transport.
> >> > > >
> >> > > >
> >> > > > > -Danny
> >> > > > >
> >> > > > > > Date: Thu=2C 31 Oct 2013 13:28:41 -0700
> >> > > > > > Subject: Re: [DISCUSS] Switching Streams from Camel deployme=
nt to
> >> > > .war
> >> > > > > deployment
> >> > > > > > From: chris@cxtsoftware.com
> >> > > > > > To: dev@streams.incubator.apache.org
> >> > > > > >
> >> > > > > > Or Content Enricher [2]
> >> > > > > >
> >> > > > > > [2] http://camel.apache.org/content-enricher.html
> >> > > > > >
> >> > > > > >
> >> > > > > > On Thu=2C Oct 31=2C 2013 at 12:37 PM=2C Jason Letourneau
> >> > > > > > <jletourneau80@gmail.com>wrote:
> >> > > > > >
> >> > > > > > > check out the link here[1]
> >> > > > > > >
> >> > > > > > >
> >> > > > > > > [1]http://camel.apache.org/request-reply.html
> >> > > > > > >
> >> > > > > > > On Thu=2C Oct 31=2C 2013 at 2:32 PM=2C Danny Sullivan <
> >> > > > > dsullivan7@hotmail.com>
> >> > > > > > > wrote:
> >> > > > > > > > I have a quick Camel question that I arrived at in the
> >> > > > > implementation of
> >> > > > > > > these new components:
> >> > > > > > > >
> >> > > > > > > > Lets say I have a method in streams-activity.jar that ne=
eds
> >> all
> >> > > > > > > subscribers in the database. This would require a call to =
the
> >> > > > > > > streams-persistence.jar. So far=2C I've seen camel used mo=
stly
> >> for
> >> > > > > passing
> >> > > > > > > data through the application=2C but not for making a singl=
e
> >> > > > > request-reponse
> >> > > > > > > from within a method. How can I use Camel to get a list of=
 all
> >> > > > > subscribers
> >> > > > > > > in the streams-persistence.jar from the streams-activity.j=
ar?
> >> > > > > > > >
> >> > > > > > > >> From: dsullivan7@hotmail.com
> >> > > > > > > >> To: dev@streams.incubator.apache.org
> >> > > > > > > >> Subject: RE: [DISCUSS] Switching Streams from Camel
> >> deployment
> >> > > to
> >> > > > > .war
> >> > > > > > > deployment
> >> > > > > > > >> Date: Thu=2C 31 Oct 2013 08:40:44 -0400
> >> > > > > > > >>
> >> > > > > > > >> Excellent=2C I'll write up something as a proof of conc=
ept and
> >> > we
> >> > > > can
> >> > > > > > > discuss further to make sure everything is vanilla.
> >> > > > > > > >>
> >> > > > > > > >> > Date: Wed=2C 30 Oct 2013 17:01:28 -0400
> >> > > > > > > >> > Subject: Re: [DISCUSS] Switching Streams from Camel
> >> > deployment
> >> > > > to
> >> > > > > > > .war deployment
> >> > > > > > > >> > From: jletourneau80@gmail.com
> >> > > > > > > >> > To: dev@streams.incubator.apache.org
> >> > > > > > > >> >
> >> > > > > > > >> > That sounds pretty promising to me.
> >> > > > > > > >> >
> >> > > > > > > >> > On Wed=2C Oct 30=2C 2013 at 2:38 PM=2C Danny Sullivan=
 <
> >> > > > > > > dsullivan7@hotmail.com> wrote:
> >> > > > > > > >> > > Thanks for the feedback. You have an interesting po=
int
> >> > about
> >> > > > the
> >> > > > > > > url linking to a separate processing space. Let me tie my
> >> answer
> >> > > into
> >> > > > > your
> >> > > > > > > last question about "advocating for the simplicity at
> >> > registration
> >> > > to
> >> > > > > give
> >> > > > > > > up flexibility at registration=2C but retaining the inner =
"guts"
> >> of
> >> > > > > > > EIP/messaging". Consider a new architecture:
> >> > > > > > > >> > >
> >> > > > > > > >> > > streams-web.war: single entry point to application=
=2C but
> >> > > > > functions
> >> > > > > > > ONLY as an entry point. From here Camel routes the incomin=
g
> >> > > requests
> >> > > > > to 4
> >> > > > > > > separate jarssubscriber-registration.jar: subscriber
> >> registration
> >> > > > > > > publisher-registration.jar: publisher registrationactivity=
.jar:
> >> > > > returns
> >> > > > > > > activity (also contains subscriber warehouse and storm act=
ivity
> >> > > > > > > aggregator)publish.jar: publishes
> >> activitystreams-cassandra.jar:
> >> > > the
> >> > > > > above
> >> > > > > > > 4 jars would all have a hook into this jar which would fun=
ction
> >> > as
> >> > > a
> >> > > > > hook
> >> > > > > > > onto the database. Each jar would have camel route output =
to
> >> this
> >> > > > jar.
> >> > > > > > > >> > >
> >> > > > > > > >> > > In this implementation=2C Camel would no longer be =
the
> >> entry
> >> > > and
> >> > > > > exit
> >> > > > > > > point of a client to the application=2C but would handle t=
he
> >> > > > > communication
> >> > > > > > > between components. The flow of activity through the
> >> application
> >> > > > would
> >> > > > > be
> >> > > > > > > method based in each jar. This would allow deployment on u=
p to
> >> 6
> >> > > > > different
> >> > > > > > > process spaces. However=2C this does not address that ther=
e is a
> >> > > single
> >> > > > > > > server entry point=2C but I'm not sure if it was a concern=
 in the
> >> > > first
> >> > > > > place.
> >> > > > > > > >> > >
> >> > > > > > > >> > > My argument=2C at its basis=2C is that we should mo=
ve away
> >> > from
> >> > > > > using
> >> > > > > > > Camel as the entry point to the application. I would be ha=
ppy
> >> to
> >> > > > > maintain
> >> > > > > > > messaging between components.
> >> > > > > > > >> > >
> >> > > > > > > >> > >> Date: Wed=2C 30 Oct 2013 12:32:55 -0400
> >> > > > > > > >> > >> Subject: Re: [DISCUSS] Switching Streams from Came=
l
> >> > > > deployment
> >> > > > > to
> >> > > > > > > .war deployment
> >> > > > > > > >> > >> From: jletourneau80@gmail.com
> >> > > > > > > >> > >> To: dev@streams.incubator.apache.org
> >> > > > > > > >> > >>
> >> > > > > > > >> > >> An interesting use case that I am holding onto is =
the
> >> > > ability
> >> > > > > for
> >> > > > > > > >> > >> publishers to register via a single URL (registrat=
ion
> >> > > > > endpoint)=2C
> >> > > > > > > but
> >> > > > > > > >> > >> be sent a URL back to post to a different process =
space
> >> > for
> >> > > > > actual
> >> > > > > > > >> > >> publishing.  The same is true on the subscriber fr=
ont.
> >> > > > >  Currently=2C
> >> > > > > > > the
> >> > > > > > > >> > >> Camel/EIP infrastructure abstracts this because
> >> different
> >> > > > > > > components
> >> > > > > > > >> > >> deployed in different process spaces handling the =
route
> >> > > > > creation
> >> > > > > > > can
> >> > > > > > > >> > >> just be bolted onto a running Streams instance wit=
hout
> >> > new
> >> > > > > > > subs/pubs
> >> > > > > > > >> > >> behaving any differently than existing.  This seem=
s to
> >> > be a
> >> > > > > > > >> > >> potentially critical scaling point.  Is there a wa=
y to
> >> do
> >> > > > this
> >> > > > > with
> >> > > > > > > >> > >> the Spring solution?
> >> > > > > > > >> > >>
> >> > > > > > > >> > >> The persistence point is a good one=2C though I wo=
uld
> >> > > classify
> >> > > > > that
> >> > > > > > > as
> >> > > > > > > >> > >> "not implemented" vs "not possible" (not that you
> >> were).
> >> > > > > > > >> > >>
> >> > > > > > > >> > >> I'm not married to Camel=2C I just like the EIP ap=
proach
> >> to
> >> > > > > building
> >> > > > > > > >> > >> something that is ultimately a messaging system.  =
There
> >> > are
> >> > > > > known
> >> > > > > > > >> > >> patterns that solve at least a subset of the probl=
ems
> >> > > Streams
> >> > > > > is
> >> > > > > > > >> > >> trying to solve and implementations that can handl=
e the
> >> > > load
> >> > > > > and
> >> > > > > > > I'll
> >> > > > > > > >> > >> reiterate flexibility =3D=3D complexity almost alw=
ays.
> >> > > > > > > >> > >>
> >> > > > > > > >> > >> It comes right back to the central question: Do yo=
u
> >> want
> >> > > > > > > flexibility
> >> > > > > > > >> > >> or simplicity?  It doesn't have to be black and wh=
ite
> >> > > either
> >> > > > I
> >> > > > > > > don't
> >> > > > > > > >> > >> think...
> >> > > > > > > >> > >>
> >> > > > > > > >> > >> More pointedly: Where should we give up flexibilit=
y for
> >> > > > > > > simplicity?  I
> >> > > > > > > >> > >> read that Danny is advocating for the simplicity a=
t
> >> > > > > registration to
> >> > > > > > > >> > >> give up flexibility at registration=2C but retaini=
ng the
> >> > > inner
> >> > > > > > > "guts" of
> >> > > > > > > >> > >> EIP/messaging?  Thoughts?
> >> > > > > > > >> > >>
> >> > > > > > > >> > >> On Wed=2C Oct 30=2C 2013 at 11:37 AM=2C Danny Sull=
ivan <
> >> > > > > > > dsullivan7@hotmail.com> wrote:
> >> > > > > > > >> > >> > My argument is not for the IoC pattern as that c=
an be
> >> > > (and
> >> > > > > has
> >> > > > > > > been) implemented alongside Camel. My main argument is tha=
t the
> >> > > > syntax
> >> > > > > at
> >> > > > > > > the entry point is not only familiar but much simpler. Thi=
s
> >> > > wouldn't
> >> > > > > be a
> >> > > > > > > very strong argument if the Camel implementation wasn't mu=
ch
> >> more
> >> > > > > > > complicated but I feel that it is the case. Also=2C lookin=
g
> >> toward
> >> > > the
> >> > > > > > > future=2C if the server is restarted=2C in-routes are lost=
 in
> >> Camel.
> >> > > The
> >> > > > > way to
> >> > > > > > > curb this is to persist the dynamic routes that Camel crea=
tes=2C
> >> > and
> >> > > > > then on
> >> > > > > > > start up pull every one of these routes and recreate a dyn=
amic
> >> > > route
> >> > > > > for
> >> > > > > > > each one. Not only is this much easier to implement using =
the
> >> > > Spring
> >> > > > > web
> >> > > > > > > implementation=2C but it already has been implemented and =
you can
> >> > try
> >> > > > it
> >> > > > > by
> >> > > > > > > checking out the webservice branch=2C registering a subscr=
iber=2C
> >> > > > > restarting
> >> > > > > > > tomcat=2C and using the same url you had before. This will=
 allow
> >> > > > > subscribers
> >> > > > > > > to hang on to their urls once they register. (the same is =
true
> >> > for
> >> > > > > > > publishers: you can post via the same url after restarting
> >> > tomcat)
> >> > > > > > > >> > >> >
> >> > > > > > > >> > >> >> Date: Wed=2C 30 Oct 2013 11:00:21 -0400
> >> > > > > > > >> > >> >> Subject: Re: [DISCUSS] Switching Streams from C=
amel
> >> > > > > deployment
> >> > > > > > > to .war deployment
> >> > > > > > > >> > >> >> From: jletourneau80@gmail.com
> >> > > > > > > >> > >> >> To: dev@streams.incubator.apache.org
> >> > > > > > > >> > >> >>
> >> > > > > > > >> > >> >> To be fair=2C while the current implementation =
is
> >> > heavily
> >> > > > > > > camel-based=2C
> >> > > > > > > >> > >> >> all of the interfaces related to Streams
> >> functionality
> >> > > are
> >> > > > > not.
> >> > > > > > >  The
> >> > > > > > > >> > >> >> current model maps to what Matt has outlined in=
 my
> >> > > > opinion=2C
> >> > > > > > > though
> >> > > > > > > >> > >> >> packing names etc. probably don't follow that e=
xact
> >> > > > pattern.
> >> > > > > > > >> > >> >>
> >> > > > > > > >> > >> >> With regards to the complexity and different
> >> > components
> >> > > in
> >> > > > > the
> >> > > > > > > >> > >> >> registration process=2C this was a cut at the
> >> > abstraction
> >> > > > > based
> >> > > > > > > on the
> >> > > > > > > >> > >> >> assumption that different implementations may b=
e
> >> > plugged
> >> > > > in
> >> > > > > and
> >> > > > > > > in
> >> > > > > > > >> > >> >> fact may live on different processor space (ie.=
 a
> >> > > polling
> >> > > > > > > publisher vs
> >> > > > > > > >> > >> >> a push publisher may be instantiated on differe=
nt
> >> > > servers
> >> > > > > but
> >> > > > > > > the
> >> > > > > > > >> > >> >> registration URL is staticly defined).
> >> > > > > > > >> > >> >>
> >> > > > > > > >> > >> >> Is the main argument I am seeing for Spring  th=
e
> >> > > > > familiarity of
> >> > > > > > > its
> >> > > > > > > >> > >> >> IoC pattern implementation and syntax at the en=
try
> >> > > point?
> >> > > > > > > >> > >> >>
> >> > > > > > > >> > >> >> On Wed=2C Oct 30=2C 2013 at 10:53 AM=2C Danny S=
ullivan <
> >> > > > > > > dsullivan7@hotmail.com> wrote:
> >> > > > > > > >> > >> >> > Could you clarify whether the same entry poin=
ts
> >> > would
> >> > > > > exist
> >> > > > > > > for the camel implementation of the core (implementing the
> >> > > "process"
> >> > > > > > > method/ using a DynammicRouteBuilder) or would the webserv=
ice
> >> be
> >> > > the
> >> > > > > sole
> >> > > > > > > entry point to Streams and after it enters would it hand i=
t off
> >> > to
> >> > > > > Camel?
> >> > > > > > > And what would be the entry point for the Storm implementa=
tion?
> >> > > > > > > >> > >> >> > -Danny
> >> > > > > > > >> > >> >> >
> >> > > > > > > >> > >> >> >> Date: Wed=2C 30 Oct 2013 10:13:04 -0400
> >> > > > > > > >> > >> >> >> Subject: Re: [DISCUSS] Switching Streams fro=
m
> >> Camel
> >> > > > > > > deployment to .war deployment
> >> > > > > > > >> > >> >> >> From: m.ben.franklin@gmail.com
> >> > > > > > > >> > >> >> >> To: dev@streams.incubator.apache.org
> >> > > > > > > >> > >> >> >>
> >> > > > > > > >> > >> >> >> On Tue=2C Oct 29=2C 2013 at 2:55 PM=2C Danny=
 Sullivan <
> >> > > > > > > dsullivan7@hotmail.com>wrote:
> >> > > > > > > >> > >> >> >>
> >> > > > > > > >> > >> >> >> > My case for switching from OSGi is for
> >> simplicity
> >> > > in
> >> > > > > > > design. To follow the
> >> > > > > > > >> > >> >> >> > path of an activity through Streams in the
> >> > > > webservice=2C
> >> > > > > > > there is one main
> >> > > > > > > >> > >> >> >> > things the developer needs to understand:
> >> > > > > > > >> > >> >> >> > The @RequestMapping annotation specifies t=
he
> >> HTTP
> >> > > > entry
> >> > > > > > > point
> >> > > > > > > >> > >> >> >> > There are 4 @RequestMapping annotations th=
at
> >> > > > > correspond to
> >> > > > > > > each of the 4
> >> > > > > > > >> > >> >> >> > ways a user enters the application:
> >> registering a
> >> > > > > > > publisher=2C registering a
> >> > > > > > > >> > >> >> >> > subscriber=2C publishing activity=2C and g=
etting an
> >> > > > > activity
> >> > > > > > > stream. Where
> >> > > > > > > >> > >> >> >> > these are located in the source code can b=
e
> >> found
> >> > > by
> >> > > > > > > searching for the
> >> > > > > > > >> > >> >> >> > paths specified in the documentation (sear=
ch
> >> for
> >> > > > > > > "/publisherRegister"=2C
> >> > > > > > > >> > >> >> >> > "/subscriberRegister"=2C "/publishActivity=
"=2C
> >> > > > > "/getActivity"
> >> > > > > > > which will all
> >> > > > > > > >> > >> >> >> > lead you to StreamsWebController.java). Fr=
om
> >> the
> >> > > > > methods
> >> > > > > > > that process
> >> > > > > > > >> > >> >> >> > requests=2C the flow through the applicati=
on is
> >> > > through
> >> > > > > > > methods which can be
> >> > > > > > > >> > >> >> >> > understood by most Java programmers.
> >> > > > > > > >> > >> >> >> > The flow of activities through the current
> >> trunk
> >> > > > > branch is
> >> > > > > > > understood as
> >> > > > > > > >> > >> >> >> > follows:
> >> > > > > > > >> > >> >> >> > The string "/publisher/register" (the entr=
y
> >> point
> >> > > to
> >> > > > > > > register a publisher
> >> > > > > > > >> > >> >> >> > specified in the documentation) is the val=
ue of
> >> > the
> >> > > > > > > >> > >> >> >> > consumer.registrationEndpoint property def=
ined
> >> in
> >> > > > > > > streams.propertiesThe
> >> > > > > > > >> > >> >> >> > camelContext.xml specifies an endpoint wit=
h the
> >> > id
> >> > > > > > > >> > >> >> >> > consumerRegistrationEndpoint to have a uri
> >> equal
> >> > to
> >> > > > the
> >> > > > > > > propertyThe
> >> > > > > > > >> > >> >> >> > consumerRegistrationEndpoint routes from u=
ri
> >> > > > > > > direct:publisher register with
> >> > > > > > > >> > >> >> >> > the bean activityRegistrationProcessor nes=
ted
> >> in
> >> > > > > between
> >> > > > > > > the routeThe
> >> > > > > > > >> > >> >> >> > streams-eip-applicationContext contains a =
bean
> >> > with
> >> > > > > the id
> >> > > > > > > >> > >> >> >> > activityRegistrationProcessor created for =
the
> >> > class
> >> > > > > > > >> > >> >> >> > ActivityPublisherRegistrationProcessorThe
> >> > exchange
> >> > > > will
> >> > > > > > > enter the "process"
> >> > > > > > > >> > >> >> >> > method of the class
> >> > > > > > > ActivityPublisherRegisitrationProcessor and that this
> >> > > > > > > >> > >> >> >> > is because this class implements the
> >> "Processor"
> >> > > > > interface
> >> > > > > > > provided by
> >> > > > > > > >> > >> >> >> > CamelThe direct:add-publisher-route takes =
the
> >> > > > exchange
> >> > > > > > > output from the
> >> > > > > > > >> > >> >> >> > "process" method and routes it to the
> >> > > > > > > activityRegistrationProcessor
> >> > > > > > > >> > >> >> >> > "register" method. The bean
> >> > > > > activityRegistrationProcessor
> >> > > > > > > is defined in the
> >> > > > > > > >> > >> >> >> > streams-eip-osgi-component-import.xmlThe o=
utput
> >> > > from
> >> > > > > this
> >> > > > > > > method is then
> >> > > > > > > >> > >> >> >> > sent to the "createNewRouteForConsumer" me=
thod
> >> of
> >> > > > > > > activityConsumerRouter.
> >> > > > > > > >> > >> >> >> > This method creates a new route for the ne=
wly
> >> > > > > registered
> >> > > > > > > publisher using
> >> > > > > > > >> > >> >> >> > the private static final class
> >> > > > > DynamicConsumerRouteBuilder
> >> > > > > > > which is
> >> > > > > > > >> > >> >> >> > required to extend RouteBuilder which is
> >> provided
> >> > > by
> >> > > > > > > Camel. This
> >> > > > > > > >> > >> >> >> > DynamicConsumerRouteBuilder contains sever=
al
> >> > > methods:
> >> > > > > > > >> > >> >> >> > getConsumerReceiveMethod() (which correspo=
nds
> >> to
> >> > > > @Value
> >> > > > > > > >> > >> >> >> > ${consumer.receiveMethod} which correspond=
s to
> >> > > > > "receive")=2C
> >> > > > > > > >> > >> >> >> > getConsumerSplitMethod() (@Value
> >> > > > > > > ${consumer.splitMethod}=2C"split")=2C and
> >> > > > > > > >> > >> >> >> > getConsumerActivityQUri() (@Value
> >> > > > > ${consumerActivityQUri}=2C
> >> > > > > > > >> > >> >> >> > "direct:activityQ"). This is different tha=
n the
> >> > > > > > > camelContext.xml in that
> >> > > > > > > >> > >> >> >> > the route is being created programmaticall=
y.
> >> What
> >> > > > this
> >> > > > > is
> >> > > > > > > doing is routing
> >> > > > > > > >> > >> >> >> > input from the inroute url (which Camel do=
es
> >> > > > > automatically
> >> > > > > > > through the
> >> > > > > > > >> > >> >> >> > configure method which is required to be
> >> > > overridden)=2C
> >> > > > > to
> >> > > > > > > the "receive"
> >> > > > > > > >> > >> >> >> > method of ActivityConsumer=2C then to the =
"split"
> >> > > > method
> >> > > > > of
> >> > > > > > > ActivityConsumer=2C
> >> > > > > > > >> > >> >> >> > and then to the "direct:activityQ" which i=
f you
> >> > > look
> >> > > > > back
> >> > > > > > > in the
> >> > > > > > > >> > >> >> >> > camelContext.xml routes to the
> >> > > > > "activemq:queue:activities"
> >> > > > > > > which then
> >> > > > > > > >> > >> >> >> > routes to "receiveExchange"
> >> > > > > > > >> > >> >> >> > This is the process to register a publishe=
r.
> >> The
> >> > > > > process
> >> > > > > > > for registering a
> >> > > > > > > >> > >> >> >> > subscriber is relatively the same though i=
t
> >> > > involves
> >> > > > > > > separate classes with
> >> > > > > > > >> > >> >> >> > their own private static final RouteBuilde=
r
> >> > class.
> >> > > > > From my
> >> > > > > > > perspective=2C the
> >> > > > > > > >> > >> >> >> > two most difficult things with setting thi=
s
> >> > project
> >> > > > up
> >> > > > > > > were understanding
> >> > > > > > > >> > >> >> >> > that the "process" method of the class tha=
t
> >> > > > implements
> >> > > > > > > "Processor" is the
> >> > > > > > > >> > >> >> >> > entry point and the DynamicConsumerRouteBu=
ilder
> >> > > > creates
> >> > > > > > > the second entry
> >> > > > > > > >> > >> >> >> > point (The 5th and last points). This made=
 the
> >> > > > project
> >> > > > > > > very=2C VERY hard to
> >> > > > > > > >> > >> >> >> > understand.
> >> > > > > > > >> > >> >> >> > In addition to simplicity of design=2C the=
 mvn
> >> > clean
> >> > > > > install
> >> > > > > > > of the web
> >> > > > > > > >> > >> >> >> > service project is much faster and small s=
cale
> >> > > > activity
> >> > > > > > > publishing is also
> >> > > > > > > >> > >> >> >> > faster (see my email about load testing). =
These
> >> > are
> >> > > > > minor
> >> > > > > > > points though as
> >> > > > > > > >> > >> >> >> > compilation has no effect on deployment. O=
SGi
> >> > does
> >> > > > add
> >> > > > > the
> >> > > > > > > benefit of
> >> > > > > > > >> > >> >> >> > modularized programming which is valuable=
=2C
> >> > though I
> >> > > > > think
> >> > > > > > > the added
> >> > > > > > > >> > >> >> >> > complexity of Camel merits moving the proj=
ect
> >> > away
> >> > > > from
> >> > > > > > > this paradigm.
> >> > > > > > > >> > >> >> >> >
> >> > > > > > > >> > >> >> >>
> >> > > > > > > >> > >> >> >> I agree that the project is pretty difficult=
 to
> >> > > > > understand
> >> > > > > > > ATM.  I think
> >> > > > > > > >> > >> >> >> what we need to do is think about what the
> >> > > > > responsibilities
> >> > > > > > > of the code are
> >> > > > > > > >> > >> >> >> and allow for different implementations that=
 are
> >> > not
> >> > > so
> >> > > > > > > tightly coupled as
> >> > > > > > > >> > >> >> >> they are now.  For instance=2C having worked=
 with
> >> > Storm
> >> > > > to
> >> > > > > > > ingest millions of
> >> > > > > > > >> > >> >> >> activities a day=2C I personally would like =
to see
> >> > > > streams
> >> > > > > be
> >> > > > > > > responsible for
> >> > > > > > > >> > >> >> >> defining an over-arching orchestration model=
 that
> >> > can
> >> > > > be
> >> > > > > > > implemented within
> >> > > > > > > >> > >> >> >> a single war or on top of a distributed syst=
em.
> >> >  This
> >> > > > > would
> >> > > > > > > look something
> >> > > > > > > >> > >> >> >> like the follows:
> >> > > > > > > >> > >> >> >>
> >> > > > > > > >> > >> >> >> |__ Streams-Core  (Base classes=2C interface=
s=2C
> >> > > utilities=2C
> >> > > > > > > extensions=2C etc)
> >> > > > > > > >> > >> >> >> |
> >> > > > > > > >> > >> >> >> |__ Streams-Storm (Storm implementation of t=
he
> >> > core)
> >> > > > > > > >> > >> >> >> |
> >> > > > > > > >> > >> >> >> |__ Streams-Camel (Camel implementation of t=
he
> >> > core)
> >> > > > > > > >> > >> >> >> |
> >> > > > > > > >> > >> >> >> |__ Streams-WS (Web service implementation)
> >> > > > > > > >> > >> >> >>
> >> > > > > > > >> > >> >> >>
> >> > > > > > > >> > >> >> >> > Danny
> >> > > > > > > >> > >> >> >> >
> >> > > > > > > >> > >> >> >> >
> >> > > > > > > >> > >> >> >> > > From: dsullivan7@hotmail.com
> >> > > > > > > >> > >> >> >> > > To: dev@streams.incubator.apache.org
> >> > > > > > > >> > >> >> >> > > Subject: [DISCUSS] Switching Streams fro=
m
> >> Camel
> >> > > > > > > deployment to .war
> >> > > > > > > >> > >> >> >> > deployment
> >> > > > > > > >> > >> >> >> > > Date: Tue=2C 29 Oct 2013 11:07:39 -0400
> >> > > > > > > >> > >> >> >> > >
> >> > > > > > > >> > >> >> >> > > The discussion thread for switching Stre=
ams
> >> > from
> >> > > > > > > Camel/osgi/Servicemix
> >> > > > > > > >> > >> >> >> > to a single .war deployment
> >> > > > > > > >> > >> >> >> >
> >> > > > > > > >> > >> >> >> >
> >> > > > > > > >> > >> >> >
> >> > > > > > > >> > >> >
> >> > > > > > > >> > >
> >> > > > > > > >>
> >> > > > > > > >
> >> > > > > > >
> >> > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
 		 	   		  =

--_96523367-9983-4688-8ef4-e757902a84aa_--