camel-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From conflue...@apache.org
Subject [CONF] Apache Camel > Stream caching
Date Wed, 13 Feb 2013 11:41:00 GMT
<html>
<head>
    <base href="https://cwiki.apache.org/confluence">
            <link rel="stylesheet" href="/confluence/s/2042/9/1/_/styles/combined.css?spaceKey=CAMEL&amp;forWysiwyg=true"
type="text/css">
    </head>
<body style="background: white;" bgcolor="white" class="email-body">
<div id="pageContent">
<div id="notificationFormat">
<div class="wiki-content">
<div class="email">
    <h2><a href="https://cwiki.apache.org/confluence/display/CAMEL/Stream+caching">Stream
caching</a></h2>
    <h4>Page <b>edited</b> by             <a href="https://cwiki.apache.org/confluence/display/~ay">Aki
Yoshida</a>
    </h4>
        <br/>
                         <h4>Changes (2)</h4>
                                 
    
<div id="page-diffs">
                    <table class="diff" cellpadding="0" cellspacing="0">
    
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >| CamelCachedOutputStreamThreshold
| 64kb | Size in bytes when the stream should be spooled to disk instead of keeping in memory.
Use a value of 0 or negative to disable it all together so streams is always kept in memory
regardless of their size. | <br>| CamelCachedOutputStreamOutputDirectory | java.io.tmpdir
| Base directory where temporary files for spooled streams should be stored. | <br></td></tr>
            <tr><td class="diff-added-lines" style="background-color: #dfd;">|
CamelCachedOutputStreamCipherTransformation | null | *Camel 2.11.0:* If set, the temporary
files are encrypted using the specified cipher transformation. | <br></td></tr>
            <tr><td class="diff-unchanged" > <br>You set these properties
on the CamelContext as shown below: <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
            <tr><td class="diff-unchanged" >context.getProperties().put(CachedOutputStream.THRESHOLD,
&quot;1024&quot;); <br>context.getProperties().put(CachedOutputStream.BUFFER_SIZE,
&quot;4096&quot;); <br></td></tr>
            <tr><td class="diff-added-lines" style="background-color: #dfd;">//
to enable encryption using RC4 <br>// context.getProperties().put(CachedOutputStream.CIPHER_TRANSFORMATION,
&quot;RC4&quot;); <br></td></tr>
            <tr><td class="diff-unchanged" >{code} <br> <br></td></tr>
            <tr><td class="diff-snipped" >...<br></td></tr>
    
            </table>
    </div>                            <h4>Full Content</h4>
                    <div class="notificationGreySide">
        <h2><a name="Streamcaching-Streamcaching"></a>Stream caching</h2>
<p>While stream types (like StreamSource, InputStream and Reader) are commonly used
in messaging for performance reasons, they also have an important drawback: they can only
be read once.  In order to be able to work with message content multiple times, the stream
needs to be cached.</p>

<p>Streams are caching in memory. In Camel 2.0, large stream messages (over 64 Kb) will
be cached in a temporary file instead &#8211; Camel itself will handle deleting the temporary
file once the cached stream is no longer necessary.</p>

<p>In Camel 2.0 stream cache is default <b>disabled</b> out of the box.<br/>
In Camel 1.x stream cache is default <b>enabled</b> out of the box.</p>

<div class='panelMacro'><table class='infoMacro'><colgroup><col width='24'><col></colgroup><tr><td
valign='top'><img src="/confluence/images/icons/emoticons/information.gif" width="16"
height="16" align="absmiddle" alt="" border="0"></td><td><b>StreamCache
Affects your payload object</b><br />The StreamCache will affect your payload
object as it will replace the Stream payload with a <tt>org.apache.camel.StreamCache</tt>
object.<br/>
This <tt>StreamCache</tt> is capable of being re-readable and thus possible to
better be routed within Camel using redelivery or <a href="/confluence/display/CAMEL/Content+Based+Router"
title="Content Based Router">Content Based Router</a> or the likes.

<p>However to not change the payload under the covers without the end user really knowing
we changed the default in Camel 2.0 to <b>disabled</b>. So in Camel 2.0 you have
to explicit enable it if you want to use it.</p></td></tr></table></div>

<h3><a name="Streamcaching-Enablingstreamcaching"></a>Enabling stream caching</h3>

<p>In Apache Camel, you can explicitly enable stream caching for a single route with
the <b><tt>streamCaching</tt></b> DSL method:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
from(<span class="code-quote">"jbi:service:http:<span class="code-comment">//foo.bar.org/MyService"</span>)
</span>    .streamCaching()
    .to(<span class="code-quote">"jbi:service:http:<span class="code-comment">//foo.bar.org/MyOtherService"</span>);</span>
</pre>
</div></div>

<p>In Spring XML you enable it by setting the <tt>streamCache="true"</tt>
attribute on the <tt>route</tt> tag.</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-xml">
<span class="code-tag">&lt;route streamCache=<span class="code-quote">"true"</span>&gt;</span>
   <span class="code-tag">&lt;from uri=<span class="code-quote">"jbi:service:http://foo.bar.org/MyService"</span>/&gt;</span>
   <span class="code-tag">&lt;to uri=<span class="code-quote">"jbi:service:http://foo.bar.org/MyOtherService"</span>/&gt;</span>
<span class="code-tag">&lt;/route&gt;</span>
</pre>
</div></div>

<h3><a name="Streamcaching-Scopes"></a>Scopes</h3>
<p>StreamCache supports the global and per route scope. So by setting the streamCache
attribute on camelContext you can enable/disable it globally.</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-xml">
<span class="code-tag">&lt;camelContext streamCache=<span class="code-quote">"true"</span>&gt;</span>
   ...
<span class="code-tag">&lt;/camelContext&gt;</span>
</pre>
</div></div>

<p>The route scope is configured by the <tt>streamCache</tt> attribute on
the <tt>&lt;route&gt;</tt> tag such as:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-xml">
<span class="code-tag">&lt;route streamCache=<span class="code-quote">"true"</span>&gt;</span>
   <span class="code-tag">&lt;from uri=<span class="code-quote">"jbi:service:http://foo.bar.org/MyService"</span>/&gt;</span>
   <span class="code-tag">&lt;to uri=<span class="code-quote">"jbi:service:http://foo.bar.org/MyOtherService"</span>/&gt;</span>
<span class="code-tag">&lt;/route&gt;</span>
</pre>
</div></div>

<p>You can mix and match for instance you can enable it globally and disable it on a
particular route such as:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-xml">
<span class="code-tag">&lt;camelContext streamCache=<span class="code-quote">"true"</span>&gt;</span>
  <span class="code-tag">&lt;route&gt;</span>
    <span class="code-tag">&lt;from uri=<span class="code-quote">"jbi:service:http://foo.bar.org/MyService"</span>/&gt;</span>
    <span class="code-tag">&lt;to uri=<span class="code-quote">"jbi:service:http://foo.bar.org/MyOtherService"</span>/&gt;</span>
  <span class="code-tag">&lt;/route&gt;</span>
  
  <span class="code-tag">&lt;route streamCache=<span class="code-quote">"false"</span>&gt;</span>
    <span class="code-tag">&lt;from uri=<span class="code-quote">"jms:queue:foo"</span>/&gt;</span>
    <span class="code-tag">&lt;to uri=<span class="code-quote">"jms:queue:bar"</span>/&gt;</span>
  <span class="code-tag">&lt;/route&gt;</span>

<span class="code-tag">&lt;/camelContext&gt;</span>
</pre>
</div></div>

<h4><a name="Streamcaching-EnablingfromJavaDSL"></a>Enabling from Java DSL</h4>
<p>In Camel 2.0 you can enable stream cache by setting the property on CamelContext,
for instance in a RouteBuilder class:</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
  context.setStreamCache(<span class="code-keyword">true</span>);

  from(<span class="code-quote">"jbi:service:http:<span class="code-comment">//foo.bar.org/MyService"</span>)
</span>     .to(<span class="code-quote">"jbi:service:http:<span class="code-comment">//foo.bar.org/MyOtherService"</span>);</span>
</pre>
</div></div>

<h3><a name="Streamcaching-Disablestreamcachingexplicitly"></a>Disable stream
caching explicitly</h3>
<p>If you have enabled stream caching globally on the CamelContext and you want to disable
it for certain routes in a selective manner, you can use the following syntax.</p>

<p><b>Spring DSL:</b></p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-xml">
<span class="code-tag">&lt;camelContext streamCache=<span class="code-quote">"true"</span>&gt;</span>
  <span class="code-tag">&lt;route streamCache=<span class="code-quote">"false"</span>&gt;</span>
    <span class="code-tag">&lt;from uri=<span class="code-quote">"jetty:http://0.0.0.0:9090"</span>/&gt;</span>
    <span class="code-tag">&lt;to uri=<span class="code-quote">"file:target/incoming"</span>/&gt;</span>
  <span class="code-tag">&lt;/route&gt;</span>
<span class="code-tag">&lt;/camelContext&gt;</span>
</pre>
</div></div>

<p><b>Java DSL:</b></p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
  context.setStreamCache(<span class="code-keyword">true</span>);

  from(<span class="code-quote">"jetty:http:<span class="code-comment">//0.0.0.0:9090"</span>).noStreamCaching()
</span>     .to(<span class="code-quote">"file:target/incoming"</span>);
</pre>
</div></div>

<h3><a name="Streamcaching-Implicitlyenabledformulticastanddeadletterchannel%28Camel1.x%29"></a>Implicitly
enabled for multicast and dead letter channel (Camel 1.x)</h3>
<p>Some EIPs require that the message content can be read multiple times. Stream caching
will be automatically enabled when using these EIPs in your routes:</p>
<ul>
	<li><a href="/confluence/display/CAMEL/Multicast" title="Multicast">Multicast
</a> will implicitly cache streams to ensure that all the endpoints can access the message
content</li>
	<li><a href="/confluence/display/CAMEL/Dead+Letter+Channel" title="Dead Letter Channel">Dead
Letter Channel </a> uses stream caching to ensure that the message content can actually
be read again when redelivering a message</li>
</ul>



<h3><a name="Streamcaching-Streamingcachetofiles"></a>Streaming cache to
files</h3>
<p>When stream cache is enabled it will by default spool big streams to files instead
of keeping them in memory. The default threshold is 64kb but you can configure it with the
following properties:</p>

<div class='table-wrap'>
<table class='confluenceTable'><tbody>
<tr>
<th class='confluenceTh'> Property </th>
<th class='confluenceTh'> Default </th>
<th class='confluenceTh'> Description </th>
</tr>
<tr>
<td class='confluenceTd'> CamelCachedOutputStreamBufferSize </td>
<td class='confluenceTd'> 2kb </td>
<td class='confluenceTd'> <b>Camel 2.9.4, 2.10.2, 2.11.0:</b> Size in bytes
of the buffer used in the stream. </td>
</tr>
<tr>
<td class='confluenceTd'> CamelCachedOutputStreamThreshold </td>
<td class='confluenceTd'> 64kb </td>
<td class='confluenceTd'> Size in bytes when the stream should be spooled to disk instead
of keeping in memory. Use a value of 0 or negative to disable it all together so streams is
always kept in memory regardless of their size. </td>
</tr>
<tr>
<td class='confluenceTd'> CamelCachedOutputStreamOutputDirectory </td>
<td class='confluenceTd'> java.io.tmpdir </td>
<td class='confluenceTd'> Base directory where temporary files for spooled streams should
be stored. </td>
</tr>
<tr>
<td class='confluenceTd'> CamelCachedOutputStreamCipherTransformation </td>
<td class='confluenceTd'> null </td>
<td class='confluenceTd'> <b>Camel 2.11.0:</b> If set, the temporary files
are encrypted using the specified cipher transformation. </td>
</tr>
</tbody></table>
</div>


<p>You set these properties on the CamelContext as shown below:</p>

<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
context.getProperties().put(CachedOutputStream.TEMP_DIR, <span class="code-quote">"/tmp/cachedir"</span>);
context.getProperties().put(CachedOutputStream.THRESHOLD, <span class="code-quote">"1024"</span>);
context.getProperties().put(CachedOutputStream.BUFFER_SIZE, <span class="code-quote">"4096"</span>);
<span class="code-comment">// to enable encryption using RC4
</span><span class="code-comment">// context.getProperties().put(CachedOutputStream.CIPHER_TRANSFORMATION,
<span class="code-quote">"RC4"</span>);</span>
</pre>
</div></div>

<h4><a name="Streamcaching-Disablingspoolingtodisk"></a>Disabling spooling
to disk</h4>
<p><b>Available as of *Camel 1.6.2/2.0</b><br/>
You can disable spooling to disk by setting a threshold of 0 or a negative value.</p>
<div class="code panel" style="border-width: 1px;"><div class="codeContent panelContent">
<pre class="code-java">
<span class="code-comment">// disable spooling to disk
</span>context.getProperties().put(CachedOutputStream.THRESHOLD, <span class="code-quote">"0"</span>);
</pre>
</div></div>

<h3><a name="Streamcaching-Howitworks%3F"></a>How it works?</h3>
<p>In order to determine if a type requires caching, we leverage the <a href="/confluence/display/CAMEL/Type+Converter"
title="Type Converter">type converter </a> feature.  Any type that requires stream
caching can be converted into an <b><tt>org.apache.camel.StreamCache</tt></b>
instance.</p>
    </div>
        <div id="commentsSection" class="wiki-content pageSection">
        <div style="float: right;">
            <a href="https://cwiki.apache.org/confluence/users/viewnotifications.action"
class="grey">Change Notification Preferences</a>
        </div>
        <a href="https://cwiki.apache.org/confluence/display/CAMEL/Stream+caching">View
Online</a>
        |
        <a href="https://cwiki.apache.org/confluence/pages/diffpagesbyversion.action?pageId=112456&revisedVersion=13&originalVersion=12">View
Changes</a>
                |
        <a href="https://cwiki.apache.org/confluence/display/CAMEL/Stream+caching?showComments=true&amp;showCommentArea=true#addcomment">Add
Comment</a>
            </div>
</div>
</div>
</div>
</div>
</body>
</html>

Mime
View raw message