camel-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matt Pavlovich (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (CAMEL-5466) new virtual endpoint component
Date Mon, 30 Jul 2012 18:25:35 GMT

    [ https://issues.apache.org/jira/browse/CAMEL-5466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13425094#comment-13425094
] 

Matt Pavlovich commented on CAMEL-5466:
---------------------------------------

Hi-

I see more customers finding the XML-based route as their preference for a number of reasons.
I'm seeing clients heading there, because it is easier to manage, and you can have processes
that output a XML-based route right into a deploy folder. That is a lot easier to manage than
writing out some Java, compile+deploy. I'm also hearing the "Fuse IDE" as a reason to go XML-based.

I can see this being very useful in two scenarios:

1. Similar to Ari's, where we have 1,000's of queues doing simple q->q processing and message
volume isn't a huge issue.

2. ActiveMQ migrations, where you need to drain all the queues in one system and move it into
the new one.

My concern with #1 would be how to handle latency from looping over 1,000's of endpoints.
I see the receive timeout of 100ms, and that would be pretty slow to complete for a couple
thousand queues. Is there a way to leverage already existing Camel thread pools to split up
some of this work?  Perhaps, have 4-10 core threads that split up the 1,000 as equally as
possible to lower the latency a bit?

Very interesting component, thanks Ari!


                
> new virtual endpoint component
> ------------------------------
>
>                 Key: CAMEL-5466
>                 URL: https://issues.apache.org/jira/browse/CAMEL-5466
>             Project: Camel
>          Issue Type: Improvement
>    Affects Versions: 2.9.2
>            Reporter: Ari Mando
>             Fix For: 2.9.3
>
>         Attachments: camel-virtual.tar.gz
>
>
> I had the recurring need for a virtual endpoint in camel routes. Often I need to define
the same route, but just with different endpoint locations, typically queues.
> For example the following simple route will move messages between queues:
>     <route>
>         <from uri="virtual://source-queue?real=activemq://@?username=username@amp;password=password"/>
>         <to uri="virtual://destination-queue?real=activemq://@?username=username@amp;password=password"/>
>     </route>
> Then in a virtual.properties file you would list the substitution template values:
> *source-queue,destination-queue
> queue1-in,queue1-out
> queue2-in,queue2-out
> queue3-in,queue3-out
> The above properties file would be like adding 3 routes, the first being like:
>  <route>
>         <from uri="virtual://source-queue?real=activemq://queue1-in?username=username@amp;password=password"/>
>         <to uri="virtual://destination-queue?real=activemq2://queue1-out?username=username@amp;password=password"/>
>     </route>
> The supplied impl is a bit rough, it needs some more work. The first cut was sufficient
for the usecase I needed it for.
> I already thought it should be really like:
> <virtual id="test" uri="file://queues.xml">
>     <route>
>         <from uri="virtual://source-queue?real=amq://@?"/>
>         <to uri="virtual://destination-queue?real=amq2://@?"/>
>     </route>
> </virtual>
> then an xml file:
> <endpointValues>
>     <header>
>         <value>source-queue</value>
>         <value>destination-queue</value>
>     </headers>
>     <entry>
>         <value>queue1,queue4</value>
>         <value>queue5,queue6</value>
>     </entry>
> </endpointValues> 
> Using the DSL would allow multiple virtual routes with different config being pulled
from other camel endpoints. 
> Dynamic routing! Call it virtual routing or template routing.
> Tasks to be finished on it:
> * implement threading (concurrentConsumers) and other scaling concerns
> * move to high level dsl model
> * unit tests
> * source configuration from enrichment uri call
> * more robust error handling
> * handling of ScheduledConsumers,async etc
> * expression (regex style) uri templating (current only @ value substituion)

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira

        

Mime
View raw message