camel-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Claus Ibsen <claus.ib...@gmail.com>
Subject Re: Camel 2.3 - Default threading profile - Your view
Date Fri, 19 Mar 2010 10:07:45 GMT
Hi

You can see notes here as well
http://camel.apache.org/camel-23-threadpool-configuration.html


On Thu, Mar 18, 2010 at 2:24 PM, Ashwin Karpe <askarpe@progress.com> wrote:
>
> Hi Claus,
>
> I do like the settings as initial values which could be potentially
> overridden by the components. I also agree with the workQueue filling up
> being the trigger to add more threads.
>
> It would be great if all components could uniformly apply these initial
> values which could ten be changed by the application/component users
> themselves. This will prevent components from starving each other out with
> their individual settings
>

Moving forward Components can leverage
CamelContext.getExecutorServiceStrategy() to create the kind of thread
pool they want.
But using this strategy you let Camel aid you and have it
- export the pool for JMX so it can be managed
- handle lifecycle by shutting it down when Camel shutdown
- logging details about the created pool
- allows 3rd party strategies to hook in and use their own thread pool
factory (eg WorkManager from a J2EE server)
- easier to create thread pool than the ugly ThreadPoolExecutor constructor

When time permits I will create a wiki page with more details.
Its also being documented in chapter 10 of the Camel in Action book as well.



> Hope this helps.
>
> Cheers,
>
> Ashwin...
>
>
> Claus Ibsen-2 wrote:
>>
>> Hi
>>
>> I am posting this to user so Camel end users have a chance to make an
>> impact on this matter.
>>
>>
>> I am working on making configuring thread pools in Camel much easier.
>> https://issues.apache.org/activemq/browse/CAMEL-1588
>>
>> In light of this work we are introducing a ThreadPoolProfile which
>> contains settings to be used when a thread pool is to be created.
>>
>> The idea currently is that there is a *default* profile that is used
>> normally, and its has the following options
>>     private Integer poolSize = 10;
>>     private Integer maxPoolSize = 20;
>>     private Long keepAliveTime = 60L;
>>     private TimeUnit timeUnit = TimeUnit.SECONDS;
>>     private Integer maxQueueSize = 1000;
>>     private ThreadPoolRejectedPolicy rejectedPolicy;
>>
>> And the rejectedPolicy will default to use: CallerRuns. There are 4
>> options: Abort, CallerRuns, DiscardOldest, Discard;
>>
>>
>> I am currently wondering if these default settings is matching what
>> you would expect / like to be default in Camel 2.3 onwards.
>> How this works in Java is that the thread pool will have as min 10
>> threads in the pool and a worker queue with a 1000 capacity.
>> And only when its exhausted, eg the capacity has been hit, it will
>> grow the pool up till 20 threads to help.
>> And if threads is idle for more than 60 seconds it will be terminated.
>>
>> There are two options which I may want to adjust change
>> - maxQueueSize = 1000
>> - rejectedPolicy = CallerRuns
>>
>> Is the default capacity of 1000 to high? For example having 100, will
>> let the thread pool to grow sooner to help out.
>> Or is 100 to low. Also mind that the task queue is in memory so if the
>> server crashes those tasks are gone.
>>
>> And the rejectedPolicy with CallerRuns allows to let the caller thread
>> execute the task instead of handing it over to the thread pool.
>> This is like a last resort if the thread pool is really exhausted and
>> otherwise would have to either abort or discard a task.
>> However the good point is that by doing this its automatically
>> throttling the caller as it cannot at the same time send new tasks to
>> the thread pool.
>>
>> We could also add a new option: Wait. Which lets the caller wait a bit
>> to see if the task queue has room to handover the task. But when
>> waiting you
>> have to cater for timeout to avoid potentially waiting for a long
>> time. And this requires custom code to write, as the existing 4
>> options is provided
>> out of the box from JDK.
>>
>>
>> So when are these thread pools being used?
>> - EIPs (splitter, aggregator, multicast, wiretap etc. etc.)
>> - ProducerTemplate having a cached pool of Producer
>> - Threads DSL
>> - toAsync DSL
>>
>> The various components may have their own thread pool, such as Jetty,
>> Mina, Spring JMS etc.
>>
>>
>> Any thoughts?
>>
>>
>>
>>
>> --
>> Claus Ibsen
>> Apache Camel Committer
>>
>> Author of Camel in Action: http://www.manning.com/ibsen/
>> Open Source Integration: http://fusesource.com
>> Blog: http://davsclaus.blogspot.com/
>> Twitter: http://twitter.com/davsclaus
>>
>>
>
>
> -----
> ---
> Ashwin Karpe, Principal Consultant, PS - Opensource Center of Competence
> Progress Software Corporation
> 14 Oak Park Drive
> Bedford, MA 01730
> ---
> +1-972-304-9084 (Office)
> +1-972-971-1700 (Mobile)
> ----
> Blog: http://opensourceknowledge.blogspot.com/
>
>
> --
> View this message in context: http://old.nabble.com/Camel-2.3---Default-threading-profile---Your-view-tp27943985p27945489.html
> Sent from the Camel - Users mailing list archive at Nabble.com.
>
>



-- 
Claus Ibsen
Apache Camel Committer

Author of Camel in Action: http://www.manning.com/ibsen/
Open Source Integration: http://fusesource.com
Blog: http://davsclaus.blogspot.com/
Twitter: http://twitter.com/davsclaus

Mime
View raw message