camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shirazi ...@farhad.eu>
Subject Re: ThrottlingInflightRoutePolicy does not take effect in DSL
Date Sun, 29 Nov 2009 14:17:28 GMT

Hmmm... ok. Then i don't get what the scope attribute of the RoutePolicy
should mean. The doc. says:
"scope (Route | Context) : A scope for either Route or Context which defines
if the current number of inflight exchanges is context based or for that
particular route."
I would expect that by defining the scope as "Context" then the total number
of exchanges in all the routes defined in the context would be taken into
account by the throttling policy.

-Farhad


Claus Ibsen-2 wrote:
> 
> On Sun, Nov 29, 2009 at 2:57 PM, shirazi <me@farhad.eu> wrote:
>>
>> Well, the example I've depicted is a simplified version of a much more
>> complicated flow. The scenario i've tried to test is such that: there are
>> a
>> couple of hundred files in the start folder, and I want to ensure that
>> there
>> are max. 3 files in all the subroutes in the context at any given time.
>> So
>> what i expect is that after appr. 3-4 seconds there would be 3 files in
>> proceed1 and proceed2 folders, and before at least one of them end up in
>> the
>> archive folder (would take 20 seconds), the throttling policy should stop
>> the file consumer to read more files in and hence no files should be
>> coming
>> in the proceed folders in the meantime.
>> But it is not what's happening. The files keep coming into the proceed
>> folders and after 20 seconds I have around 20 files in the proceed
>> folders
>> ??! Do you have any idea?
>>
> 
> Well its metrics used for the throttling are *in flight* which means
> Exchange objects currently inside CamelContext being routed (in the
> JVM memory).
> When an Exchange is done (= complete) such as in your case when it
> have been written to a proceed folder, then that Exchange is removed
> from memory,
> and therefore also do no longer count as an *in flight* Exchange.
> 
> You have to create your own logic if the throttling should be based on
> number of files listed in the proceed folders.
> 
> 
>> -Farhad
>>
>>
>> Claus Ibsen-2 wrote:
>>>
>>> On Sun, Nov 29, 2009 at 11:17 AM, shirazi <me@farhad.eu> wrote:
>>>>
>>>> I'm not sure i quite understand what you mean. Are you suggesting that
>>>> in
>>>> order to have the throttling policy to kick in, I should make the route
>>>> threaded/concurrent ?!
>>>
>>> You are using a file consumer which is using a single thread by
>>> default to consume files, and hence there will only be at most 1 in
>>> flight exchange in progress.
>>>
>>> Then you have a multicast where you send the file to 2 endpoints. the
>>> multicast is by default non parallel either. If you enable
>>> parallelProcessing() on the multicast you would then have 2 exchanges
>>> in flight, one for each of the multicasted destinations. However those
>>> destinations dont do much only write the file to another folder. So
>>> everything happens so fast.
>>>
>>> The dynamic throttler in this example dont make much sense as there
>>> are just not enough messages concurrently being routed. Instead you
>>> either have to
>>> - use more threads for processing concurrent files using threads (see
>>> the blog post)
>>> - create a lot of files so there are enough files to process
>>>
>>> Or try this example which is based on JMS where its more common to
>>> have a lot of messages being routed
>>> http://camel.apache.org/route-throttling-example.html
>>>
>>>
>>>
>>>>
>>>> -Farhad
>>>>
>>>>
>>>> Claus Ibsen-2 wrote:
>>>>>
>>>>> On Sun, Nov 29, 2009 at 9:32 AM, shirazi <me@farhad.eu> wrote:
>>>>>>
>>>>>> Hi,
>>>>>> Please have a look at the following route def.:
>>>>>>
>>>>>> RouteBuilder:
>>>>>> ==========
>>>>>>
>>>>>> fromF("file:///tmp/start").routeId("test.route.1").routePolicyRef("myPolicy").
>>>>>>        multicast().delay(1000).
>>>>>>                to("file:///tmp/proceed1").
>>>>>>                to("file:///tmp/proceed2");
>>>>>>
>>>>>>
>>>>>> from("file:///tmp/proceed2").routeId("test.route.2").routePolicyRef("myPolicy").delay(20000).
>>>>>>        to("file:///tmp/archive");
>>>>>>
>>>>>> applicationContext.xml
>>>>>> ==========
>>>>>>    <bean id="myPolicy"
>>>>>> class="org.apache.camel.impl.ThrottlingInflightRoutePolicy">
>>>>>>        <property name="scope" value="Context"/>
>>>>>>        <property name="maxInflightExchanges" value="3"/>
>>>>>>        <property name="resumePercentOfMax" value="40"/>
>>>>>>        <property name="loggingLevel" value="WARN"/>
>>>>>>    </bean>
>>>>>>
>>>>>> The throttling policy does not take effect, since files keep coming
>>>>>> into
>>>>>> the
>>>>>> tmp/proceed dirs, regardless
>>>>>> the policy that says no more than 3 files at a time. Now I am clearly
>>>>>> doing
>>>>>> something wrong, but according
>>>>>> to the doc. what I have here should be enough.
>>>>>>
>>>>>
>>>>> No the route is fine. The file consumer is not concurrent, and neither
>>>>> is the multicast by default.
>>>>>
>>>>> Read this blog entry
>>>>> http://davsclaus.blogspot.com/2009/05/on-road-to-camel-20-concurrency-with.html
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>> Regards,
>>>>>> Farhad S.
>>>>>> --
>>>>>> View this message in context:
>>>>>> http://old.nabble.com/ThrottlingInflightRoutePolicy-does-not-take-effect-in-DSL-tp26560144p26560144.html
>>>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Claus Ibsen
>>>>> Apache Camel Committer
>>>>>
>>>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>>>> Open Source Integration: http://fusesource.com
>>>>> Blog: http://davsclaus.blogspot.com/
>>>>> Twitter: http://twitter.com/davsclaus
>>>>>
>>>>>
>>>>
>>>> --
>>>> View this message in context:
>>>> http://old.nabble.com/ThrottlingInflightRoutePolicy-does-not-take-effect-in-DSL-tp26560144p26560485.html
>>>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>>>
>>>>
>>>
>>>
>>>
>>> --
>>> Claus Ibsen
>>> Apache Camel Committer
>>>
>>> Author of Camel in Action: http://www.manning.com/ibsen/
>>> Open Source Integration: http://fusesource.com
>>> Blog: http://davsclaus.blogspot.com/
>>> Twitter: http://twitter.com/davsclaus
>>>
>>>
>>
>> --
>> View this message in context:
>> http://old.nabble.com/ThrottlingInflightRoutePolicy-does-not-take-effect-in-DSL-tp26560144p26562271.html
>> Sent from the Camel - Users mailing list archive at Nabble.com.
>>
>>
> 
> 
> 
> -- 
> Claus Ibsen
> Apache Camel Committer
> 
> Author of Camel in Action: http://www.manning.com/ibsen/
> Open Source Integration: http://fusesource.com
> Blog: http://davsclaus.blogspot.com/
> Twitter: http://twitter.com/davsclaus
> 
> 

-- 
View this message in context: http://old.nabble.com/ThrottlingInflightRoutePolicy-does-not-take-effect-in-DSL-tp26560144p26562427.html
Sent from the Camel - Users mailing list archive at Nabble.com.


Mime
View raw message