camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Matt Pavlovich <mattr...@gmail.com>
Subject Re: Question on the proper usage of Camel
Date Fri, 10 Feb 2012 15:21:55 GMT
Hi Grzegorz-

What you are experiencing is not uncommon for folks that are used to 
100% Java-based coding.  Its not really a technical issue, as much as it 
is a conceptual one.  I think one of the most important concepts to 
think about is that the Camel approach encourages "configuring", vs 
"coding".  If you have a lot of existing Java code, it can be difficult 
to make the decision to scrap it, but ultimately, I believe that you 
will find you'll have far less project artifacts to maintain when using 
Camel vs writing your own.  Also, consider the upgrade cycle.  When you 
write your own transport connectors, you have to maintain and re-build 
that code when upgrades for the underlying technology changes.  With 
Camel, you just have to keep up with any changes to the Camel API 
(usually, very minor).  As Camel upgrades, it brings the updated 
dependencies with it-- at little or no impact to your Camel routes.

A lot of this comes down to maintainability and speed at which you can 
implement an integration solution-- less project artifacts equals faster 
delivery, easier maintenance.

1. If you are not using the components, you are missing out on one of 
the most important benefit Camel brings.  All that connection setup and 
tear down code is redundant.  If you are using JMS, HTTP, and databases, 
your Java code is going to have a lot of redundancy to what Camel 
provides for you.  If you have some custom protocol that Camel doesn't 
support yet, consider writing a small wrapper "entry service" that uses 
a protocol that Camel does support.  A Web Service, or a JMS queue.  At 
the end of the day, less code is always easier to support.

2. If you are making a lot of direct calls, you may consider the 
benefits of having *more* JMS in the solution.  JMS provides an easy and 
reliable way to exchange information, and the guaranteed delivery, error 
handling, HA and load balancing capabilites are provided for you, so you 
don't have to worry about coding up process to account for those 
scenarios.  Using JMS would also provide you some statistics and metrics 
"for free".  Design your integrations around queues, and you can know 
exactly how many requests were handled by process X.

3. Perhaps, replacing the "direct" calls with JMS Queues would help to 
some degree.  Also, look into the Camel tracer and log component.  The 
?showAll=true option on the log component is gold.  It shows you all the 
Camel exchange headers, properties and message payload.  Great for 
debugging.

4. I haven't done any specific load tests against the direct endpoint, 
but I haven't seen anything that leads me to believe its horribly slow 
or inefficient.  A stack of method calls doesn't always necessarily mean 
its inefficient.

5. See #3.  One of the concepts I try to convey is that instead of 
thinking of an integration from a big picture standpoint, think of it in 
terms of discreet synchronous and asynchronous steps.  These "steps" are 
your routes.  Leveraging queues and topics is a good way to provide 
separation of steps and routes.  Ultimately, there will be a few giant 
routes that are just a pain.  For those, leveraging the tracer, and log 
steps makes debugging those a snap.

Hope this helps,
Matt Pavlovich

On 2/10/12 5:34 AM, Grzegorz Borkowski wrote:
> Hi All,
>
> In our project we've been using Camel for a year, but with very mixed
> feelings. I'd like to ask you if the way we use it is correct - maybe our
> problems come from the wrong usage of the tool. Camel experts, please share
> your opinions on this. Below you'll find description of the project.
>
>
> Out project is an integration tool, integrating 3 other systems, call them
> A, B and C. We receive the message from system A, query for some data from
> system B and send the message to system C. Camel seemed to be ideal tool
> for such scenario.
> In practice, there are couple of problems though.
>
> 1. It turned out that we don't use any typical communication channels to
> integrate with the systems. All those systems provided us with a library
> (jar) which is simply a stub, which presents us with an API to call remote
> systems synchronously. Even if those stubs use JMS or other channels
> underneath, we don't interact with them directly. The only thing we do, is
> just calling simple synchronous Java method on the stub. Something similar
> to the old Java RMI. Thus, the whole power of Camel components is of no
> value for us.
>
> 2. There is one place where we use Camel JMS component - we have a test
> input, for receiving test data, which uses JMS communication. So currently
> this is the only place in the code we use Camel components for real
> integration with external systems. We're also now developing additional
> interface, and this will be the second entry point based on JMS. So those
> are only places when we use Camel capabilities.
>
> 3. Inside the application, we have a workflow build on the Camel routers,
> predicates, and processors. All of them are glued by the "direct"
> components. So when XML message comes, we pass it to a router, which passes
> it to a proper processor, then to next router etc. There are around 20
> processors and 10 routers in the flow. In practice however, we see only one
> advantage of using Camel for this: ability to generate visual diagram of
> the flow (using camel:dot feature). There are many downsides though. First,
> there are many couplings between the processors, so we end up passing tens
> of headers in the messages between processors. We already call it
> "header-oriented programming" or simply "header hell". It's a nightmare.
> Then, it's horror to debug the code when for example the router passed the
> message to the wrong path. If it was a normal java code written by us, with
> "if then else", we could just put a breakpoint there and debug it. But in
> camel, if you configure routers using the built-in predicates, like:
> from(...).choide().when(header(..).isEqualsTo(...)).to(...) - then you're
> not able to debug it easily, because the if-then-else logic at runtime is
> performed by camel engine, not our code.
>
> 4. Using the "direct" component connections everywhere seems to be
> extremely inefficient. Say I have a flow: message goes through 2 routers
> and two processors, linked by direct components. If you generate stack
> trace at the end of this path, you will see literally hundreds of camel
> calls. Where do they come from? It's especially frustrating when you try to
> debug it - it's infeasible. I hate debugging Spring proxies because they
> insert about 5-10 framework method calls between my classes. Camel does the
> same, but it uses hundreds of framework calls instead. The stack trace is
> littered with DelegateAsyncProcessor and AsyncProcessorHelper calls. I
> wouldn't really be surprised if I saw StackOverflowError somewhere in the
> flow.
>
> 5. Testing the Camel flow is not a nice experience as well. For every test,
> you have to set up those dummy endpoints, and record expected number of
> incoming messages. Then, when there is an exception thrown in the flow,
> instead of just breaking the test where it was thrown, we see the cryptic
> message at the end "expected receiving 1 message, received 0". Then you
> have to analyze the log to find what was the real reason. Also, Camel
> waiting for the messages, make the tests much slower, because  time is lost
> on this waiting.
>
>
> Thoughts? Recommendations? What do we do wrong?
>

Mime
View raw message