camel-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject [CONF] Apache Camel > Camel 2.3 - ThreadPool Configuration
Date Fri, 12 Mar 2010 11:17:00 GMT
    <base href="">
            <link rel="stylesheet" href="/confluence/s/1519/1/1/_/styles/combined.css?spaceKey=CAMEL&amp;forWysiwyg=true"
<body style="background-color: white" bgcolor="white">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
     <h2><a href="">Camel
2.3 - ThreadPool Configuration</a></h2>
     <h4>Page <b>edited</b> by             <a href="">Claus
     <div class="notificationGreySide">
         <h2><a name="Camel2.3-ThreadPoolConfiguration-DesignNotesforThreadPoolConfiguration"></a>Design
Notes for ThreadPool Configuration</h2>

<p><a href="" rel="nofollow">CAMEL-1588</a>
is the ticket for a new and improved thread pool configuration for Apache Camel. Its intended
for Camel 2.3.</p>

<h3><a name="Camel2.3-ThreadPoolConfiguration-Scope"></a>Scope</h3>

<p>Camel uses thread pool in various places such as <a href="/confluence/display/CAMEL/EIP"
title="EIP">EIP</a> patterns, <a href="/confluence/display/CAMEL/Components" title="Components">Components</a>,
<a href="/confluence/display/CAMEL/Async" title="Async">Async</a> API and whatnot.
The aim is to improved and allow easier to configure those thread pools in a more streamlined
manner. The goal is to offer both a fine grained configuration where you can tweak individual
pools and have more coarse grained configuration with fallback to <em>global</em>
settings etc.</p>

<h4><a name="Camel2.3-ThreadPoolConfiguration-Outsidescope"></a>Outside

<p>Some components provides their own thread pool configuration and management which
Camel of course cannot and should not try to tailor with. For example <a href="/confluence/display/CAMEL/Jetty"
title="Jetty">Jetty</a> is such an example.</p>

<h3><a name="Camel2.3-ThreadPoolConfiguration-UsagesofthreadpoolsinCamel"></a>Usages
of thread pools in Camel</h3>

<p>Currently Camel uses thread pools in <b>camel-core</b> in the following
<ul class="alternate" type="square">
	<li><tt>DefaultComponent</tt> - Optional thread pool for components in
need of such</li>
	<li><tt>DefaultEndpoint</tt> - Optional thread pool for endpoints in need
of such</li>
	<li><tt>DefaultProducerTemplate</tt> - Used by the <a href="/confluence/display/CAMEL/Async"
title="Async">Async</a> API of this template</li>
	<li><tt>ScheduledPollConsumer</tt> - Needs a <tt>ScheduledExecutorService</tt>
to schedule its tasks</li>
	<li><tt>RecipientListDefinition</tt> - The <a href="/confluence/display/CAMEL/Recipient+List"
title="Recipient List">Recipient List</a> EIP pattern</li>
	<li><tt>SplitDefinition</tt> - The <a href="/confluence/display/CAMEL/Splitter"
title="Splitter">Splitter</a> EIP pattern</li>
	<li><tt>ThreadsDefinition</tt> - The <tt>threads</tt> DSL</li>
	<li><tt>ToDefinition</tt> - Used by the <tt>toAsync</tt> variation</li>
	<li><tt>WireTapDefinition</tt> - The <a href="/confluence/display/CAMEL/Wire+Tap"
title="Wire Tap">Wire Tap</a> EIP pattern</li>
	<li><tt>MulticastProcessor</tt> - The underlying thread pool</li>
	<li><tt>OnCompletionProcessor</tt> - For sending async on completion routes</li>
	<li><tt>SendAsyncProcessor</tt> - The <tt>toAsync</tt> processor
	<li><tt>ThreadsProcessor</tt> - The underlying thread pool</li>
	<li><tt>WireTapProcessor</tt> - For sending async wire taps</li>
	<li><tt>SedaConsumer</tt> - To support the <tt>concurrentConsumers</tt>
and <tt>multipleConsumers</tt> options (uses separate pools)</li>

<h3><a name="Camel2.3-ThreadPoolConfiguration-Existingconfiguration"></a>Existing

<p>You can configure the thread pool using the <b>setExecutorService</b>
setter methods that usually exists on those places where its in use. Some EIP patterns offer
a <tt>executorServiceRef</tt> option to refer to some pool to be looked up in
the <a href="/confluence/display/CAMEL/Registry" title="Registry">Registry</a>.</p>

<p>We should ensure all EIPs can be configured to use a custom thread pool in a nice
and easy way. <b>DONE</b></p>

<h3><a name="Camel2.3-ThreadPoolConfiguration-UsingdefaultThreadPools"></a>Using
default ThreadPools</h3>

<p>We should use <tt>CachedThreadPool</tt> from the JDK Core as its the
best general purpose pool for many short lived tasks, which is what Camel really does. Processing
many messages in a short live.  <b>DONE</b></p>

<p>Only used SingleExecutorService for background tasks, and ScheduledExecutorService
for scheduled tasks <b>DONE</b></p>

<h3><a name="Camel2.3-ThreadPoolConfiguration-ThreadPoolscope"></a>ThreadPool

<p>It should be possible to configure a thread pool on either per CamelContext level
or per Route level, such as you can do with AutoStartup and the likes.<br/>
Then you can say, eg this route should use this pool, which have 20/50 in the pool size etc.
<b>CHANGE OF PLAN</b></p>

<h3><a name="Camel2.3-ThreadPoolConfiguration-Threadpoolconfigurationbyrules"></a>Thread
pool configuration by rules</h3>

<p>It should be possible to to define a set of rules which matches which thread pool
a given source should use.<br/>
It should be pattern based so you can say all EIPs should use this pool, all endpoints that
pool etc.</p>

<p>A ruleset something like this:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
&lt;threadPoolRule route=<span class="code-quote">"*"</span> source=<span
class="code-quote">"Aggregator"</span> executorServiceRef=<span class="code-quote">"myAggPool"</span>/&gt;
&lt;threadPoolRule route=<span class="code-quote">"*"</span> source=<span
class="code-quote">"To"</span> executorServiceRef=<span class="code-quote">"mySendPool"</span>/&gt;
&lt;threadPoolRule route=<span class="code-quote">"route3"</span> source=<span
class="code-quote">"*"</span> executorServiceRef=<span class="code-quote">"myRoute3Pool"</span>/&gt;

<p>Where it will match against route first, so if we got a route3 then it will pick
among those<br/>
It should be possible to use wildcard and reg exp in the <tt>route</tt> and <tt>source</tt>

<h3><a name="Camel2.3-ThreadPoolConfiguration-Theproblemwithshutdownandrestartingpools"></a>The
problem with shutdown and restarting pools</h3>

<p>The ExecutorService API does not allow to restart a thread pool, which is PITA. So
we need to find a better strategy for stopping vs. shutdown.<br/>
Currently when we stop we also terminate the threadpool, and then re-create it on start. This
only works for default pools which we can create again.<br/>
But for custom thread pools we have no way to create them again as the pool is already created
when its given to us. </p>

<p>We may have to only shutdown thread pools if CamelContext is stopping. And then if
end user stop a route from JMX we can keep the thread pool around.<br/>
Only issue is the scheduled pool should stop scheduling tasks, which may be a bit more trickier
to avoid.</p>

<h3><a name="Camel2.3-ThreadPoolConfiguration-TheproblemwithComponent%2CEndpoint"></a>The
problem with Component, Endpoint</h3>

<p>The DefaultComponent and DefaultEndpoint exposes API to get an ExecutorService. We
should remove these API as you should use <tt>ExecutorServiceStrategy</tt> from
<tt>CamelContext</tt> to obtain a thread pool. <b>DONE</b></p>

<h3><a name="Camel2.3-ThreadPoolConfiguration-Managedthreadpool"></a>Managed
thread pool</h3>

<p>Check whether the thread pools is managed by default and avail in JConsole. If not
we should probably at least expose some read-only data about the pools.</p>

<h3><a name="Camel2.3-ThreadPoolConfiguration-SpringFactoryforcreatingcustompools"></a>Spring
Factory for creating custom pools</h3>

<p>Create a Spring XML DSL for defining thread pools using custom options such as corePoolSize,
maxPoolSize, keepAlive, thread name etc. <b>DONE</b></p>

<h3><a name="Camel2.3-ThreadPoolConfiguration-PluggableExecutorServiceSPI"></a>Pluggable
ExecutorService SPI</h3>

<p>We need a <tt>org.apache.camel.spi.ExeuctorServieStrategy</tt> which
is pluggable so end users can plugin their own strategy how to create thread pools. They can
leverage a WorkManager API from J2EE server etc. <b>DONE</b></p>

<h3><a name="Camel2.3-ThreadPoolConfiguration-Customizablethreadname"></a>Customizable
thread name</h3>

<p>We should offer a simple pattern syntax so end users can customize the pattern the
thread name is created with: eg Something like: <tt>Camel Thread ${counter} - ${name</tt>}.
Where counter and suffix is dynamic parameters. The counter is an unique incrementing thread
counter. And name is provided from the source which as a way to name thread, such as a seda
endpoint uri. <b>DONE</b></p>
     <div id="commentsSection" class="wiki-content pageSection">
       <div style="float: right;">
            <a href=""
class="grey">Change Notification Preferences</a>

       <a href="">View
       <a href="">View
       <a href=";showCommentArea=true#addcomment">Add

View raw message