camel-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From build...@apache.org
Subject svn commit: r907746 [3/3] - in /websites/production/camel/content: book-cookbook.html book-in-one-page.html cache/main.pageCache parallel-processing-and-ordering.html
Date Fri, 02 May 2014 09:19:51 GMT
Modified: websites/production/camel/content/cache/main.pageCache
==============================================================================
Binary files - no diff available.

Modified: websites/production/camel/content/parallel-processing-and-ordering.html
==============================================================================
--- websites/production/camel/content/parallel-processing-and-ordering.html (original)
+++ websites/production/camel/content/parallel-processing-and-ordering.html Fri May  2 09:19:51
2014
@@ -84,53 +84,10 @@
 	<tbody>
         <tr>
         <td valign="top" width="100%">
-<div class="wiki-content maincontent"><h2 id="ParallelProcessingandOrdering-ParallelProcessingandOrdering">Parallel
Processing and Ordering </h2>
-
-<p>It is a common requirement to want to use parallel processing of messages for throughput
and load balancing, while at the same time process certain kinds of messages in order.</p>
-
-<h3 id="ParallelProcessingandOrdering-Howtoachieveparallelprocessing">How to achieve
parallel processing</h3>
-
-<p>You can send messages to a number of Camel <a shape="rect" href="components.html">Components</a>
to achieve parallel processing and load balancing such as</p>
-
-<ul><li><a shape="rect" href="seda.html">SEDA</a> for in-JVM load
balancing across a thread pool</li><li><a shape="rect" href="activemq.html">ActiveMQ</a>
or <a shape="rect" href="jms.html">JMS</a> for distributed load balancing and
parallel processing</li><li><a shape="rect" href="jpa.html">JPA</a>
for using the database as a poor mans message broker</li></ul>
-
-
-<p>When processing messages concurrently, you should consider ordering and concurrency
issues. These are described below</p>
-
-<h4 id="ParallelProcessingandOrdering-Concurrencyissues">Concurrency issues</h4>
-
-<p>Note that there is no concurrency or locking issue when using <a shape="rect"
href="activemq.html">ActiveMQ</a>, <a shape="rect" href="jms.html">JMS</a>
or <a shape="rect" href="seda.html">SEDA</a> by design; they are designed for
highly concurrent use. However there are possible concurrency issues in the <a shape="rect"
href="processor.html">Processor</a> of the messages i.e. what the processor does
with the message? </p>
-
-<p>For example if a processor of a message transfers money from one account to another
account; you probably want to use a database with pessimistic locking to ensure that operation
takes place atomically. </p>
-
-<h4 id="ParallelProcessingandOrdering-Orderingissues">Ordering issues</h4>
-
-<p>As soon as you send multiple messages to different threads or processes you will
end up with an unknown ordering across the entire message stream as each thread is going to
process messages concurrently. </p>
-
-<p>For many use cases the order of messages is not too important. However for some
applications this can be crucial. e.g. if a customer submits a purchase order version 1, then
amends it and sends version 2; you don't want to process the first version last (so that you
loose the update). Your <a shape="rect" href="processor.html">Processor</a> might
be clever enough to ignore old messages. If not you need to preserve order.</p>
-
-<h3 id="ParallelProcessingandOrdering-Recommendations">Recommendations</h3>
-
-<p>This topic is large and diverse with lots of different requirements; but from a
high level here are our recommendations on parallel processing, ordering and concurrency</p>
-
-<ul><li>for distributed locking, use a database by default, they are very good
at it <img class="emoticon emoticon-smile" src="https://cwiki.apache.org/confluence/s/en_GB-1988229788/4109/76e0dbb30bc8580e459c201f3535d84f9283a9ac.1/_/images/icons/emoticons/smile.png"
data-emoticon-name="smile" alt="(smile)"></li><li>to preserve ordering across
a JMS queue consider using <a shape="rect" class="external-link" href="http://activemq.apache.org/exclusive-consumer.html">Exclusive
Consumers</a> in the <a shape="rect" href="activemq.html">ActiveMQ</a> component</li><li>even
better are <a shape="rect" class="external-link" href="http://activemq.apache.org/message-groups.html">Message
Groups</a> which allows you to preserve ordering across messages while still offering
parallelisation via the <strong>JMSXGroupID</strong> header to determine what
can be parallelized</li><li>if you receive messages out of order you could use
the <a shape="rect" href="resequencer.html">Resequencer</a> to put them 
 back together again</li></ul>
-
-
-<p>A good rule of thumb to help reduce ordering problems is to make sure each single
can be processed as an atomic unit in parallel (either without concurrency issues or using
say, database locking); or if it can't, use a <a shape="rect" class="external-link" href="http://activemq.apache.org/message-groups.html">Message
Group</a> to relate the messages together which need to be processed in order by a single
thread.</p>
-
-<h3 id="ParallelProcessingandOrdering-UsingMessageGroupswithCamel">Using Message Groups
with Camel</h3>
-
-<p>To use a Message Group with Camel you just need to add a header to the output JMS
message based on some kind of <a shape="rect" href="correlation-identifier.html">Correlation
Identifier</a> to correlate messages which should be processed in order by a single
thread - so that things which don't correlate together can be processed concurrently.</p>
-
-<p>For example the following code shows how to create a message group using an XPath
expression taking an invoice's product code as the <a shape="rect" href="correlation-identifier.html">Correlation
Identifier</a></p>
-
-<div class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent
pdl">
-<script class="theme: Default; brush: java; gutter: false" type="syntaxhighlighter"><![CDATA[
-from(&quot;activemq:a&quot;).setHeader(&quot;JMSXGroupID&quot;, xpath(&quot;/invoice/productCode&quot;)).to(&quot;activemq:b&quot;);
+<div class="wiki-content maincontent"><h2 id="ParallelProcessingandOrdering-ParallelProcessingandOrdering">Parallel
Processing and Ordering</h2><p>It is a common requirement to want to use parallel
processing of messages for throughput and load balancing, while at the same time process certain
kinds of messages in order.</p><h3 id="ParallelProcessingandOrdering-Howtoachieveparallelprocessing">How
to achieve parallel processing</h3><p>You can send messages to a number of Camel
<a shape="rect" href="components.html">Components</a> to achieve parallel processing
and load balancing such as</p><ul><li><a shape="rect" href="seda.html">SEDA</a>
for in-JVM load balancing across a thread pool</li><li><a shape="rect" href="activemq.html">ActiveMQ</a>
or <a shape="rect" href="jms.html">JMS</a> for distributed load balancing and
parallel processing</li><li><a shape="rect" href="jpa.html">JPA</a>
for using the database as a poor mans message broker</li></ul><p>When processing
messages concurrent
 ly, you should consider ordering and concurrency issues. These are described below</p><h4
id="ParallelProcessingandOrdering-Concurrencyissues">Concurrency issues</h4><p>Note
that there is no concurrency or locking issue when using <a shape="rect" href="activemq.html">ActiveMQ</a>,
<a shape="rect" href="jms.html">JMS</a> or <a shape="rect" href="seda.html">SEDA</a>
by design; they are designed for highly concurrent use. However there are possible concurrency
issues in the <a shape="rect" href="processor.html">Processor</a> of the messages
i.e. what the processor does with the message?</p><p>For example if a processor
of a message transfers money from one account to another account; you probably want to use
a database with pessimistic locking to ensure that operation takes place atomically.</p><h4
id="ParallelProcessingandOrdering-Orderingissues">Ordering issues</h4><p>As
soon as you send multiple messages to different threads or processes you will end up with
an unknown ordering acro
 ss the entire message stream as each thread is going to process messages concurrently.</p><p>For
many use cases the order of messages is not too important. However for some applications this
can be crucial. e.g. if a customer submits a purchase order version 1, then amends it and
sends version 2; you don't want to process the first version last (so that you loose the update).
Your <a shape="rect" href="processor.html">Processor</a> might be clever enough
to ignore old messages. If not you need to preserve order.</p><h3 id="ParallelProcessingandOrdering-Recommendations">Recommendations</h3><p>This
topic is large and diverse with lots of different requirements; but from a high level here
are our recommendations on parallel processing, ordering and concurrency</p><ul><li>for
distributed locking, use a database by default, they are very good at it <img class="emoticon
emoticon-smile" src="https://cwiki.apache.org/confluence/s/en_GB-1988229788/4109/76e0dbb30bc8580e459c201f3535d84f9283a9a
 c.1/_/images/icons/emoticons/smile.png" data-emoticon-name="smile" alt="(smile)"></li><li>to
preserve ordering across a JMS queue consider using <a shape="rect" class="external-link"
href="http://activemq.apache.org/exclusive-consumer.html">Exclusive Consumers</a>
in the <a shape="rect" href="activemq.html">ActiveMQ</a> component</li><li>even
better are <a shape="rect" class="external-link" href="http://activemq.apache.org/message-groups.html">Message
Groups</a> which allows you to preserve ordering across messages while still offering
parallelisation via the <strong>JMSXGroupID</strong> header to determine what
can be parallelized</li><li>if you receive messages out of order you could use
the <a shape="rect" href="resequencer.html">Resequencer</a> to put them back together
again</li></ul><p>A good rule of thumb to help reduce ordering problems
is to make sure each single can be processed as an atomic unit in parallel (either without
concurrency issues or using say, database locking
 ); or if it can't, use a <a shape="rect" class="external-link" href="http://activemq.apache.org/message-groups.html">Message
Group</a> to relate the messages together which need to be processed in order by a single
thread.</p><h3 id="ParallelProcessingandOrdering-UsingMessageGroupswithCamel">Using
Message Groups with Camel</h3><p>To use a Message Group with Camel you just need
to add a header to the output JMS message based on some kind of <a shape="rect" href="correlation-identifier.html">Correlation
Identifier</a> to correlate messages which should be processed in order by a single
thread - so that things which don't correlate together can be processed concurrently.</p><p>For
example the following code shows how to create a message group using an XPath expression taking
an invoice's product code as the <a shape="rect" href="correlation-identifier.html">Correlation
Identifier</a></p><div class="code panel pdl" style="border-width: 1px;"><div
class="codeContent panelContent pdl">
+<script class="theme: Default; brush: java; gutter: false" type="syntaxhighlighter"><![CDATA[from(&quot;activemq:a&quot;).setHeader(JmsConstants.JMS_X_GROUP_ID,
xpath(&quot;/invoice/productCode&quot;)).to(&quot;activemq:b&quot;);
 ]]></script>
-</div></div>
-
-<p>You can of course use the <a shape="rect" href="xml-configuration.html">Xml
Configuration</a> if you prefer</p></div>
+</div></div><p>You can of course use the <a shape="rect" href="xml-configuration.html">Xml
Configuration</a> if you prefer</p></div>
         </td>
         <td valign="top">
           <div class="navigation">



Mime
View raw message