camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From James Strachan <james.strac...@gmail.com>
Subject Re: Abstracting Routes using Components
Date Thu, 28 Oct 2010 04:55:53 GMT
On 27 October 2010 22:08, kristofsajdak <kristof.sajdak@gmail.com> wrote:
>
> Great feedback guys, much appreciated !

Agreed!

> jstrachan wrote:
>>
>> Agreed. If you kinda squint a bit and think of the route builders as
>> 'objects' and you want to compose 2 different routes together with
>> shared dependencies its a bit like 'constructors' and 'data hiding'.
>>
>> We have protocol A and B and we need to route the output of A to the
>> input of B; so rather than B needing to know the implementation detail
>> of A; we add a constructor parameter for B's input...
>>
>> class A extends ProtocolBuilder {...}
>>
>> class B extends ProtocolBuilder {
>>   public B(String inputUri) { ... }
>> }
>>
>> // lets use Java code to simulate DI...
>> ProtocolBuilder a = new A();
>> ProtocolBuilder b = new B(b.getOutputUri());
>>
>>
>> we've now wired A and B together without B knowing anything about the
>> implementation detail of A.
>>
>> Its quite common for lots of protocols to be 'unix pipe-ish' with an
>> input and output; so we could have helper classes where there's
>> standard input/output endpoints like this to create a pipeline...
>>
>> CompositeProtocol composite = Protocols.pipeline(A.class, B.class,
>> C.class);
>>
>> where A:out -> B:in and B:out -> C:in. Indeed that could return a new
>> protocol; where composite:in maps to A:in and composite:out maps to
>> C:out
>>
>> But then this is sounding like just treating A, B and C as endpoints :)
>>
>
> Indeed that is adding more flexibility and will be quite usefull when
> nesting protocolBuilders several levels deep.
>
> However, I do have another point which relates to the specific use case of
> the customer I mentioned.
>
> Developers -> provide ProtocolBuilder implementations
> Solution consultants -> use ProtocolBuilders abstractions to wire the route.
>
> Practically, it seems to me that the easiest way of achieving that is to
> create one or more modules
> which contain the ProtocolBuilders implementation classes and the spring xml
> files which configure them.
>
> Whenever a solution consultant starts working on a new customer
> implementation he could use an archetype to create a module which has the
> dependencies to the ProtocolBuilder modules preconfigured.
>
> The solution consultant would then create the route in xml dsl and use
> camel:run to execute the route, this plugin goal would in turn bootstrap all
> the ProtocolBuilder configurations located in
> classpath*:META-INF/spring/.xml, making them available to the route at
> runtime.
>
> Now my question is: When you define a ProtocolBuilder in the spring xml,
> would it get initialized and added to the CamelContext at that point ? I
> mean I don't think it's a good idea to initialize and add all the
> ProtocolBuilders which are available in classpath*:META-INF/spring, but
> neither do I want to leave it up to the solution consultant to explicitly
> define which builders get initialized.
>
> E.g. Like with the Parameterized RouteBuilder concept I mentioned earlier
>
>  <camelContext ...
>    <routeBuilder ref="generateAndSignPdfRouteBuilder"/>
>    <routeBuilder ref="signPdfRouteBuilder"/>
>    ....
>
> Which is just way to complex for non developers in my opinion.

Agreed.

This reminds me a little bit of how we use Components in Camel. We
want by default end users to not have to do anything at all thanks to
"convention over configuration"; components come with smart defaults.
You just add the dependency for a specific component to your pom.xml
(or use the correct archetype or whatever) and you're good to go -
there may be some dependency injection configuration required if the
defaults for the component don't quite match what you need.


So a Protocol (lets use that term for now) would be a Component as
well. Sure they are more complex (there's stuff inside! :),

A Protocol may internally use Java, Spring, Scala, Ruby or Groovy DSLs
to implement itself (but thats for the implementor of the Protocol to
choose). Then a Protocol may have a default URI scheme and smart
configuration defaults - or it can be explicitly instantiated,
configured and bound to a URI scheme name (bean ID in spring for
example) via Spring / Guice / JNDI dependency injection.

It should be pretty straight forward and the actual Protocol class
should hide as much of the implementation detail as possible so the
user of the protocol doesn't need to understand the implementation;
just like with Components today.


Once that kind of hiding is possible; it should be trivial to create,
say, an archetype using protocols A and B by having, say, a little bit
of Spring XML to configure these protocols once, then non-developers
can route into and out of these protocols easily.


> It sure would be nice if you could initialize and add the ProtocolBuilder
> route configuration on demand.
> Saving overhead on startup and runtime, yet making it easy for a low tech
> guy to use all of the available
> ProtocolBuilders -> no explicit declaration of ProtocolBuilders nor any
> fiddling with dependencies in the pom.xml.

Agreed.

We can also use the same kind of tricks with Component/Endpoint as
with Protocol; i.e. if you don't use a Protocol then nothing much
happens. Even when you do use a Protocol (like using a Component),
things only really kick in when you start really using them & routing
to/from them.


> The concept of the shorthand notation could add value here as well.
>
> The global properties + tech dependencies like a datasource could be
> configured by developers using spring di in the xml. Wheras the parameters,
> which make sense in the context of the route, are defined by the solution
> consultant using the shorthand notation.
>
> E.g.
>
> ?signatureReason=JustForFun&signatureVisible=true&xslUri=com/xti/poc/camel/route/sample1/krusty.xsl


Yeah. I guess usually with Component/Endpoint type stuff, we tend to
use URI parameters to configure 'middleware type stuff'; for
Protocols, maybe the URIs should be 'protocol stuff' i.e. setting
headers which can be used inside the protocol to respond differently.

Or to say that a different way; if the endpoints into and out of a
Protocol are all a simple standard kind (say the "local" endpoints
mentioned previously), there's not going to be a whole lot of
configuration parameters for those kinds of endpoints as they are
kinda like "direct" endpoints, so we can reuse the URI parameters as
short hand for passing in configuration/header information to the
protocol?


> jstrachan wrote:
>> Thats a great point! Just like with OO, we probably need some kind of
>> data hiding; or in this case its 'endpoint hiding'. So maybe using
>> nested CamelContexts is a good thing.
>>
>> Another random idea; imagine if we had a ProtocolBuilder which created
>> a child CamelContext - so all "direct:foo" and "seda:bar" endpoints
>> were totally private. Then imagine, as a simple implementation option
>> - we map all URIs within the ProtocolBuilder's URI space to the
>> direct: endpoint within the nested CamelContext.
>>
>> So rather than having to add the "alias()" method I mentioned in the
>> other thread - we could just use URIs inside the route for the
>> 'input/output/error' URIs...
>> http://camel.465427.n5.nabble.com/implementing-Protocols-or-a-way-to-make-it-easier-to-black-box-routes-and-compose-them-with-other-ros-td3218777.html#a3218777
>>
>> e.g. imagine...
>>
>> class A extends ProtocolBuilder {
>>   public void configure() {
>>     from("direct:in").
>>      beanRef("foo").
>>      to("direct:out");
>>   }
>> }
>>
>> class B extends ProtocolBuilder {
>>   public void configure() {
>>     from("direct:in").
>>      unmarshal().jaxb().
>>      to("direct:out");
>>   }
>> }
>>
>> // now lets wire A and B
>> // kinda like "Foo | A | B | Bar" on a unix command line
>>
>> class Foo extends RouteBuilder {
>>   public void configure() {
>>     from("activemq:Foo").to("a:in");
>>     from("a:out").to("b:in");
>>     from("b:out").to("activemq:Bar");
>>   }
>> }
>>
>>
>> i.e. so A and B have no knowledge of each other (kinda like unix
>> pipes), you can compose them together in any way you like (Foo) - and
>> bind them to any middleware.
>>
>> In this example the 'local endpoints' (using direct:) are totally
>> private to each ProtocolBuilder so there's no name clashes as we're
>> using nested CamelContext's.
>>
>> Not totally sure if we should make "direct" into some magical
>> component here; or if we should just introduce some kind of "local:"
>> endpoint to use inside a ProtocolBuilder to emphasise a local,
>> internal endpoint which is private to the ProtocolBuilder? e.g.
>> replace "direct:in" to "local:in".
>>
>> The question of local v global endpoint URIs is interesting; I wonder
>> how many kinds of endpoints can even support the concept of local.
>> Using nested CamelContexts we'd have local seda/vm/direct I think;
>> though given their similarity, maybe a new 'local' endpoint is simpler
>> as it clearly describes what really is local versus what really is an
>> external/global endpoint?
>>
>> Note that there's nothing to stop these protocols using
>> external/global endpoints too; am sure they will. Its more where we
>> parameterise things together we might want to use local endpoints
>> which we can then route into / out of.
>>
>
> Having the ProtocolBuilder create a child CamelContext seems like a very
> good idea !
> This way you can use the direct: and seda endpoints with normal semantics
> and without risk of namespace collisions.
>
> Just a thought though, if you are going with this concept, wouldn't it make
> more sense to use a public: endpoint to explicitly define public endpoints ?
>
> E.g.
>
> class A extends ProtocolBuilder {
>
>  public void configure() {
>
>    from("public:in").
>     beanRef("foo").
>     to("direct:bar");
>
>     from("direct:bar").
>     to("public:out");
>
>  }
> }

Great idea :). Thinking more about this, "local" tends to sound like
"local variable" or "private" really :)


> Doesn't a local: endpoints make more sense when you are not using a nested
> Context,
> and therefore would have to define explicitly what you keep private.
> In the case of a nested CamelContext, the direct: and seda: endpoints are
> always private and defining explicitly what you want as public makes sense.

I didn't really explain it too well, let me take another stab.

A Protocol might wish to use different kinds of endpoints internally...

* middleware components (which are kinds of 'public' endpoints such as
activemq, jms, file, http etc). There is no real concept of 'hiding'
those, they are kinda real middleware stuff

* private endpoints - kinda like direct/seda but used purely inside a
protocol and private so noone else can see them

* public endpoints - used to export endpoints used inside routes to
being made public outside of the Protocol inside the
protocol/component URI schene. e.g. "public:in" inside a protocol
called "foo" would map to "foo:in" when used outside the protocol. So
a "public" endpoint is a kinda in-JVM bridge between the internals of
the protocol and the outside world.

I did wonder if we should treat things like direct/mock/seda/vm as
being 'middleware components'; as in protocols may wish to communicate
between each other or to the outside world using these kinds of
component as a kinda shared channel; then just have the public/private
endpoints when you want hiding/exporting. If we nest CamelContexts and
make direct/vm/seda purely local; there'd be no way to communicate
between protocols using those efficient non-middleware endpoints
(unless we introduce a new component that does this like "global" or
we force the global routes to route into 'public' endpoints on
protocols).

Just to throw another idea out there; maybe the mock/seda/vm/direct
type endpoints could be 'protocol nesting aware' using some naming
convention. A bit like how file systems can be absolute or relative;
maybe the URIs inside those components could have a nested-context
awareness to their names? e.g. we assume all mock/seda/vm endpoints
are absolute/global so far and we introduce a naming convention for
'private' versions of those endpoints.

But it might be we find that really to export an endpoint we want to
use 'public'; if we really want to hide an endpoint internally
'private' is all we really need; but we could go down the route of
private versions of mock/seda/vm later on if we need to?

e.g. what if "private:" prefix on a URI made a private version of that
endpoint in a nested context; so "private:foo" was syntax sugar for,
say, "private:direct:foo". Then we could have "private:seda:blah" if
we wanted to use "seda" rather than "direct"?


In summary; lots of things to think about :) but lots of stuff is
falling into place. Protocols sound like Components; we have a
consistent URI naming convention to route into and out of Protocols
and I'm liking the public/private URIs alot. Some details to be worked
out though; like do we really need nested CamelContexts's or do we
just figure out the nesting/private stuff as part of the usual URI
resolution mechanism?

-- 
James
-------
FuseSource
Email: james@fusesource.com
Web: http://fusesource.com
Twitter: jstrachan
Blog: http://macstrac.blogspot.com/

Open Source Integration

Mime
View raw message