camel-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From conflue...@apache.org
Subject [CONF] Apache Camel > HawtDB
Date Thu, 21 Oct 2010 18:36:00 GMT
<html>
<head>
    <base href="https://cwiki.apache.org/confluence">
            <link rel="stylesheet" href="/confluence/s/1810/9/1/_/styles/combined.css?spaceKey=CAMEL&amp;forWysiwyg=true"
type="text/css">
    </head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
    <h2><a href="https://cwiki.apache.org/confluence/display/CAMEL/HawtDB">HawtDB</a></h2>
    <h4>Page <b>edited</b> by             <a href="https://cwiki.apache.org/confluence/display/~davsclaus">Claus
Ibsen</a>
    </h4>
        <br/>
                         <h4>Changes (1)</h4>
                                 
    
<div id="page-diffs">
            <table class="diff" cellpadding="0" cellspacing="0">
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >[HawtDB|http://hawtdb.fusesource.org/]
is a very lightweight and embedable key value database. It allows together with Camel to provide
persistent support for various Camel features such as [Aggregator|Aggregator2].  <br>
<br></td></tr>
            <tr><td class="diff-added-lines" style="background-color: #dfd;">{warning:title=Issue
with HawtDB 1.4 or older} <br>There is a bug in HawtDB 1.4 or older which means the
filestore will not free unused space. That means the file keeps growing. This has been fixed
in HawtDB 1.5 which is shipped with Camel 2.5 onwards. <br>{warning} <br> <br></td></tr>
            <tr><td class="diff-unchanged" >Current features it provides: <br>-
HawtDBAggregationRepository <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
        </table>
</div>                            <h4>Full Content</h4>
                    <div class="notificationGreySide">
        <h2><a name="HawtDB-HawtDB"></a>HawtDB</h2>
<p><b>Available as of Camel 2.3</b></p>

<p><a href="http://hawtdb.fusesource.org/" class="external-link" rel="nofollow">HawtDB</a>
is a very lightweight and embedable key value database. It allows together with Camel to provide
persistent support for various Camel features such as <a href="/confluence/display/CAMEL/Aggregator2"
title="Aggregator2">Aggregator</a>. </p>

<div class='panelMacro'><table class='warningMacro'><colgroup><col width='24'><col></colgroup><tr><td
valign='top'><img src="/confluence/images/icons/emoticons/forbidden.gif" width="16"
height="16" align="absmiddle" alt="" border="0"></td><td><b>Issue with
HawtDB 1.4 or older</b><br />There is a bug in HawtDB 1.4 or older which means
the filestore will not free unused space. That means the file keeps growing. This has been
fixed in HawtDB 1.5 which is shipped with Camel 2.5 onwards.</td></tr></table></div>

<p>Current features it provides:</p>
<ul class="alternate" type="square">
	<li>HawtDBAggregationRepository</li>
</ul>


<h3><a name="HawtDB-UsingHawtDBAggregationRepository"></a>Using HawtDBAggregationRepository</h3>
<p><tt>HawtDBAggregationRepository</tt> is an <tt>AggregationRepository</tt>
which on the fly persists the aggregated messages. This ensures that you will not loose messages,
as the default aggregator will use an in memory only <tt>AggregationRepository</tt>.</p>

<p>It has the following options:</p>
<div class='table-wrap'>
<table class='confluenceTable'><tbody>
<tr>
<th class='confluenceTh'> Option </th>
<th class='confluenceTh'> Type </th>
<th class='confluenceTh'> Description </th>
</tr>
<tr>
<td class='confluenceTd'> <tt>repositoryName</tt> </td>
<td class='confluenceTd'> String </td>
<td class='confluenceTd'> A mandatory repository name. Allows you to use a shared <tt>HawtDBFile</tt>
for multiple repositories. </td>
</tr>
<tr>
<td class='confluenceTd'> <tt>persistentFileName</tt> </td>
<td class='confluenceTd'> String </td>
<td class='confluenceTd'> Filename for the persistent storage. If no file exists on
startup a new file is created. </td>
</tr>
<tr>
<td class='confluenceTd'> <tt>bufferSize</tt> </td>
<td class='confluenceTd'> int </td>
<td class='confluenceTd'> The size of the memory segment buffer which is mapped to the
file store. By default its 8mb. The value is in bytes. </td>
</tr>
<tr>
<td class='confluenceTd'> <tt>sync</tt> </td>
<td class='confluenceTd'> boolean </td>
<td class='confluenceTd'> Whether or not the <tt>HawtDBFile</tt> should
sync on write or not. Default is <tt>true</tt>. By sync on write ensures that
its always waiting for all writes to be spooled to disk and thus will not loose updates. If
you disable this option, then HawtDB will auto sync when it has batched up a number of writes.
</td>
</tr>
<tr>
<td class='confluenceTd'> <tt>pageSize</tt> </td>
<td class='confluenceTd'> short </td>
<td class='confluenceTd'> The size of memory pages. By default its 512 bytes. The value
is in bytes. </td>
</tr>
<tr>
<td class='confluenceTd'> <tt>hawtDBFile</tt> </td>
<td class='confluenceTd'> HawtDBFile </td>
<td class='confluenceTd'> Use an existing configured <tt>org.apache.camel.component.hawtdb.HawtDBFile</tt>
instance. </td>
</tr>
<tr>
<td class='confluenceTd'> <tt>returnOldExchange</tt> </td>
<td class='confluenceTd'> boolean </td>
<td class='confluenceTd'> Whether the get operation should return the old existing Exchange
if any existed. By default this option is <tt>false</tt> to optimize as we do
not need the old exchange when aggregating. </td>
</tr>
<tr>
<td class='confluenceTd'> <tt>useRecovery</tt> </td>
<td class='confluenceTd'> boolean </td>
<td class='confluenceTd'> Whether or not recovery is enabled. This option is by default
<tt>true</tt>. When enabled the Camel <a href="/confluence/display/CAMEL/Aggregator2"
title="Aggregator2">Aggregator</a> automatic recover failed aggregated exchange and
have them resubmitted. </td>
</tr>
<tr>
<td class='confluenceTd'> <tt>recoveryInterval</tt> </td>
<td class='confluenceTd'> long </td>
<td class='confluenceTd'> If recovery is enabled then a background task is run every
x'th time to scan for failed exchanges to recover and resubmit. By default this interval is
5000 millis. </td>
</tr>
<tr>
<td class='confluenceTd'> <tt>maximumRedeliveries</tt> </td>
<td class='confluenceTd'> int </td>
<td class='confluenceTd'> Allows you to limit the maximum number of redelivery attempts
for a recovered exchange. If enabled then the Exchange will be moved to the dead letter channel
if all redelivery attempts failed. By default this option is disabled. If this option is used
then the <tt>deadLetterUri</tt> option must also be provided. </td>
</tr>
<tr>
<td class='confluenceTd'> <tt>deadLetterUri</tt> </td>
<td class='confluenceTd'> String </td>
<td class='confluenceTd'> An endpoint uri for a <a href="/confluence/display/CAMEL/Dead+Letter+Channel"
title="Dead Letter Channel">Dead Letter Channel</a> where exhausted recovered Exchanges
will be moved. If this option is used then the <tt>maximumRedeliveries</tt> option
must also be provided. </td>
</tr>
</tbody></table>
</div>


<p>The <tt>repositoryName</tt> option must be provided. Then either the
<tt>persistentFileName</tt> or the <tt>hawtDBFile</tt> must be provided.</p>

<h3><a name="HawtDB-Whatispreservedwhenpersisting"></a>What is preserved
when persisting</h3>
<p><tt>HawtDBAggregationRepository</tt> will only preserve any <tt>Serializable</tt>
compatible data types. If a data type is not such a type its dropped and a <tt>WARN</tt>
is logged. And it only persists the <tt>Message</tt> body and the <tt>Message</tt>
headers. The <tt>Exchange</tt> properties are <b>not</b> persisted.
</p>

<h3><a name="HawtDB-Recovery"></a>Recovery</h3>
<p>The <tt>HawtDBAggregationRepository</tt> will by default recover any
failed <a href="/confluence/display/CAMEL/Exchange" title="Exchange">Exchange</a>.
It does this by having a background tasks that scans for failed <a href="/confluence/display/CAMEL/Exchange"
title="Exchange">Exchange</a>s in the persistent store. You can use the <tt>checkInterval</tt>
option to set how often this task runs. The recovery works as transactional which ensures
that Camel will try to recover and redeliver the failed <a href="/confluence/display/CAMEL/Exchange"
title="Exchange">Exchange</a>. Any <a href="/confluence/display/CAMEL/Exchange"
title="Exchange">Exchange</a> which was found to be recovered will be restored from
the persistent store and resubmitted and send out again. </p>

<p>The following headers is set when an <a href="/confluence/display/CAMEL/Exchange"
title="Exchange">Exchange</a> is being recovered/redelivered:</p>
<div class='table-wrap'>
<table class='confluenceTable'><tbody>
<tr>
<th class='confluenceTh'> Header </th>
<th class='confluenceTh'> Type </th>
<th class='confluenceTh'> Description </th>
</tr>
<tr>
<td class='confluenceTd'> <tt>Exchange.REDELIVERED</tt> </td>
<td class='confluenceTd'> Boolean </td>
<td class='confluenceTd'> Is set to true to indicate the <a href="/confluence/display/CAMEL/Exchange"
title="Exchange">Exchange</a> is being redelivered. </td>
</tr>
<tr>
<td class='confluenceTd'> <tt>Exchange.REDELIVERY_COUNTER</tt> </td>
<td class='confluenceTd'> Integer </td>
<td class='confluenceTd'> The redelivery attempt, starting from 1. </td>
</tr>
</tbody></table>
</div>


<p>Only when an <a href="/confluence/display/CAMEL/Exchange" title="Exchange">Exchange</a>
has been successfully processed it will be marked as complete which happens when the <tt>confirm</tt>
method is invoked on the <tt>AggregationRepository</tt>. This means if the same
<a href="/confluence/display/CAMEL/Exchange" title="Exchange">Exchange</a> fails
again it will be kept retried until it success.</p>

<p>You can use option <tt>maximumRedeliveries</tt> to limit the maximum
number of redelivery attempts for a given recovered <a href="/confluence/display/CAMEL/Exchange"
title="Exchange">Exchange</a>. You must also set the <tt>deadLetterUri</tt>
option so Camel knows where to send the <a href="/confluence/display/CAMEL/Exchange" title="Exchange">Exchange</a>
when the <tt>maximumRedeliveries</tt> was hit. </p>

<p>You can see some examples in the unit tests of camel-hawtdb, for example <a href="https://svn.apache.org/repos/asf/camel/trunk/components/camel-hawtdb/src/test/java/org/apache/camel/component/hawtdb/HawtDBAggregateRecoverTest.java"
class="external-link" rel="nofollow">this test</a>.</p>


<h4><a name="HawtDB-UsingHawtDBAggregationRepositoryinJavaDSL"></a>Using
HawtDBAggregationRepository in Java DSL</h4>
<p>In this example we want to persist aggregated messages in the <tt>target/data/hawtdb.dat</tt>
file.</p>

<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java"><span class="code-keyword">public</span> void configure()
<span class="code-keyword">throws</span> Exception {
    <span class="code-comment">// create the hawtdb repo
</span>    HawtDBAggregationRepository repo = <span class="code-keyword">new</span>
HawtDBAggregationRepository(<span class="code-quote">"repo1"</span>, <span
class="code-quote">"target/data/hawtdb.dat"</span>);

    <span class="code-comment">// here is the Camel route where we aggregate
</span>    from(<span class="code-quote">"direct:start"</span>)
        .aggregate(header(<span class="code-quote">"id"</span>), <span class="code-keyword">new</span>
MyAggregationStrategy())
            <span class="code-comment">// use our created hawtdb repo as aggregation
repository
</span>            .completionSize(5).aggregationRepository(repo)
            .to(<span class="code-quote">"mock:aggregated"</span>);
}
</pre>
</div></div>

<h4><a name="HawtDB-UsingHawtDBAggregationRepositoryinSpringXML"></a>Using
HawtDBAggregationRepository in Spring XML</h4>
<p>The same example but using Spring XML instead:</p>

<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-xml">
<span class="code-tag"><span class="code-comment">&lt;!-- a persistent aggregation
repository using camel-hawtdb --&gt;</span></span>
<span class="code-tag">&lt;bean id=<span class="code-quote">"repo"</span>
class=<span class="code-quote">"org.apache.camel.component.hawtdb.HawtDBAggregationRepository"</span>&gt;</span>
    <span class="code-tag"><span class="code-comment">&lt;!-- store the repo
in the hawtdb.dat file --&gt;</span></span>
    <span class="code-tag">&lt;property name=<span class="code-quote">"persistentFileName"</span>
value=<span class="code-quote">"target/data/hawtdb.dat"</span>/&gt;</span>
    <span class="code-tag"><span class="code-comment">&lt;!-- and use repo2
as the repository name --&gt;</span></span>
    <span class="code-tag">&lt;property name=<span class="code-quote">"repositoryName"</span>
value=<span class="code-quote">"repo2"</span>/&gt;</span>
<span class="code-tag">&lt;/bean&gt;</span>

<span class="code-tag"><span class="code-comment">&lt;!-- aggregate the messages
using this strategy --&gt;</span></span>
<span class="code-tag">&lt;bean id=<span class="code-quote">"myAggregatorStrategy"</span>
class=<span class="code-quote">"org.apache.camel.component.hawtdb.HawtDBSpringAggregateTest$MyAggregationStrategy"</span>/&gt;</span>

<span class="code-tag"><span class="code-comment">&lt;!-- this is the camel
routes --&gt;</span></span>
<span class="code-tag">&lt;camelContext id=<span class="code-quote">"camel"</span>
xmlns=<span class="code-quote">"http://camel.apache.org/schema/spring"</span>&gt;</span>

    <span class="code-tag">&lt;route&gt;</span>
        <span class="code-tag">&lt;from uri=<span class="code-quote">"direct:start"</span>/&gt;</span>
        <span class="code-tag"><span class="code-comment">&lt;!-- aggregate
using our strategy and hawtdb repo, and complete when we have 5 messages aggregated --&gt;</span></span>
        <span class="code-tag">&lt;aggregate strategyRef=<span class="code-quote">"myAggregatorStrategy"</span>
aggregationRepositoryRef=<span class="code-quote">"repo"</span> completionSize=<span
class="code-quote">"5"</span>&gt;</span>
            <span class="code-tag"><span class="code-comment">&lt;!-- correlate
by header with the key id --&gt;</span></span>
            <span class="code-tag">&lt;correlationExpression&gt;</span><span
class="code-tag">&lt;header&gt;</span>id<span class="code-tag">&lt;/header&gt;</span><span
class="code-tag">&lt;/correlationExpression&gt;</span>
            <span class="code-tag"><span class="code-comment">&lt;!-- send
aggregated messages to the mock endpoint --&gt;</span></span>
            <span class="code-tag">&lt;to uri=<span class="code-quote">"mock:aggregated"</span>/&gt;</span>
        <span class="code-tag">&lt;/aggregate&gt;</span>
    <span class="code-tag">&lt;/route&gt;</span>

<span class="code-tag">&lt;/camelContext&gt;</span>
</pre>
</div></div>

<h3><a name="HawtDB-Dependencies"></a>Dependencies</h3>

<p>To use <a href="/confluence/display/CAMEL/HawtDB" title="HawtDB">HawtDB</a>
in your camel routes you need to add the a dependency on <b>camel-hawtdb</b>.
</p>

<p>If you use maven you could just add the following to your pom.xml, substituting the
version number for the latest &amp; greatest release (see <a href="/confluence/display/CAMEL/Download"
title="Download">the download page for the latest versions</a>).</p>

<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-xml">
<span class="code-tag">&lt;dependency&gt;</span>
  <span class="code-tag">&lt;groupId&gt;</span>org.apache.camel<span
class="code-tag">&lt;/groupId&gt;</span>
  <span class="code-tag">&lt;artifactId&gt;</span>camel-hawtdb<span
class="code-tag">&lt;/artifactId&gt;</span>
  <span class="code-tag">&lt;version&gt;</span>2.3.0<span class="code-tag">&lt;/version&gt;</span>
<span class="code-tag">&lt;/dependency&gt;</span>
</pre>
</div></div>


<h3><a name="HawtDB-SeeAlso"></a>See Also</h3>
<ul>
	<li><a href="/confluence/display/CAMEL/Configuring+Camel" title="Configuring Camel">Configuring
Camel</a></li>
	<li><a href="/confluence/display/CAMEL/Component" title="Component">Component</a></li>
	<li><a href="/confluence/display/CAMEL/Endpoint" title="Endpoint">Endpoint</a></li>
	<li><a href="/confluence/display/CAMEL/Getting+Started" title="Getting Started">Getting
Started</a></li>
</ul>

<ul class="alternate" type="square">
	<li><a href="/confluence/display/CAMEL/Aggregator2" title="Aggregator2">Aggregator</a></li>
	<li><a href="/confluence/display/CAMEL/Components" title="Components">Components</a></li>
</ul>

    </div>
        <div id="commentsSection" class="wiki-content pageSection">
        <div style="float: right;">
            <a href="https://cwiki.apache.org/confluence/users/viewnotifications.action"
class="grey">Change Notification Preferences</a>
        </div>
        <a href="https://cwiki.apache.org/confluence/display/CAMEL/HawtDB">View Online</a>
        |
        <a href="https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=14811359&revisedVersion=14&originalVersion=13">View
Changes</a>
                |
        <a href="https://cwiki.apache.org/confluence/display/CAMEL/HawtDB?showComments=true&amp;showCommentArea=true#addcomment">Add
Comment</a>
            </div>
</div>
</div>
</div>
</div>
</body>
</html>

Mime
View raw message