camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "pmp.martins" <>
Subject How to improve efficiency with Camel?
Date Sun, 23 Feb 2014 22:05:12 GMT
I have a problem of efficiency in my project which uses Camel with the  Esper
component <>  . 

I have several external datasources feeding information to camel endpoints.
Each Camel endpoint that receives data transfers it to a route that
processes it and then delivers it at an Esper endpoint.

The image below illustrates this behavior:


The efficiency problem is that all of this is done by a single Java thread.
Thus if I have many sources, there is a huge bottleneck. 

The following code accurately illustrates what is going on with the image:


*To fix this problem, I have to handle this situation with a pool of threads
or using some sort of parallel processing.* I cannot use patterns like
multicast, recipient list, etc because all of those send the same message to
multiple endpoints / clients, which is not the case in my examples.

A possible solution would be having 1 thread per each "Datasource endpoint
-> Route -> Esper endpoint" combination, like the image bellow:


Another possible solution is to have 1 thread receive everything from the
datasources, and then dispatch it to multiple threads handling the route
processing together with the other endpoint:

*PS:* I am open to any other possible suggestions you may have.

To achieve one of these I have considered using the Camel  SEDA component
<>   component, however, this one does not
seem to allow me to have dynamic thread pools, because the
`concurrentConsumers` property is static. 

Furthermore, I was alerted in StackOverflow for loss of messages using SEDA,
so I am not sure how trustworthy this component really is. Can someone

So, at this point I am quite lost and I don't know what to do:
- Is SEDA the solution I am looking for?
- If yes, how do I integrate it with the Esper endpoint given the syntax
- Are there any other solutions / Camel components that could fix my

View this message in context:
Sent from the Camel - Users mailing list archive at

View raw message