activemq-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From build...@apache.org
Subject svn commit: r919361 - in /websites/production/activemq/content: cache/main.pageCache leveldb-store.html
Date Thu, 14 Aug 2014 19:20:42 GMT
Author: buildbot
Date: Thu Aug 14 19:20:42 2014
New Revision: 919361

Log:
Production update by buildbot for activemq

Modified:
    websites/production/activemq/content/cache/main.pageCache
    websites/production/activemq/content/leveldb-store.html

Modified: websites/production/activemq/content/cache/main.pageCache
==============================================================================
Binary files - no diff available.

Modified: websites/production/activemq/content/leveldb-store.html
==============================================================================
--- websites/production/activemq/content/leveldb-store.html (original)
+++ websites/production/activemq/content/leveldb-store.html Thu Aug 14 19:20:42 2014
@@ -85,26 +85,11 @@
                     <p class="title">Version Compatibility</p>
                             <span class="aui-icon icon-hint">Icon</span>
                 <div class="message-content">
-                            
-<p>Available in ActiveMQ 5.8.0 and newer </p>
+                            <p>Available in ActiveMQ 5.8.0 and newer</p>
                     </div>
     </div>
-
-
-<p>The LevelDB Store is a file based persistence database that is local to the message
broker that is using it. It has been optimized to provide even faster persistence than KahaDB.
 It's similar to KahahDB but instead of using a custom B-Tree implementation to index the
write ahead logs, it uses <a shape="rect" class="external-link" href="https://code.google.com/p/leveldb/"
rel="nofollow">LevelDB</a> based indexes which have several nice properties due to
the 'append only' files access patterns : </p>
-
-<ul><li>Fast updates (No need to do random disk updates)</li><li>Concurrent
reads</li><li>Fast index snapshots using hard links</li></ul>
-
-
-<p>Both KahaDB and the LevelDB store have to do periodic garbage collection cycles
to determine which log files can deleted.  In the case of KahaDB, this can be quite expensive
as you increase the amount of data stored and can cause read/write stalls while the collection
occurs.  The LevelDB store uses a much cheaper algorithm to determine when log files can be
collected and avoids those stalls.</p>
-
-<h2 id="LevelDBStore-Configuration">Configuration</h2>
-
-<p>You can configure ActiveMQ to use LevelDB for its persistence adapter - like below
:</p>
-
-<div class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent
pdl">
-<script class="theme: Default; brush: java; gutter: false" type="syntaxhighlighter"><![CDATA[
-  &lt;broker brokerName=&quot;broker&quot; ... &gt;
+<p>The LevelDB Store is a file based persistence database that is local to the message
broker that is using it. It has been optimized to provide even faster persistence than KahaDB.
It's similar to KahahDB but instead of using a custom B-Tree implementation to index the write
ahead logs, it uses <a shape="rect" class="external-link" href="https://code.google.com/p/leveldb/"
rel="nofollow">LevelDB</a> based indexes which have several nice properties due to
the 'append only' files access patterns :</p><ul><li>Fast updates (No need
to do random disk updates)</li><li>Concurrent reads</li><li>Fast index
snapshots using hard links</li></ul><p>Both KahaDB and the LevelDB store
have to do periodic garbage collection cycles to determine which log files can deleted. In
the case of KahaDB, this can be quite expensive as you increase the amount of data stored
and can cause read/write stalls while the collection occurs. The LevelDB store uses a much
cheaper algorithm to determine when log files 
 can be collected and avoids those stalls.</p><h2 id="LevelDBStore-Configuration">Configuration</h2><p>You
can configure ActiveMQ to use LevelDB for its persistence adapter - like below :</p><div
class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent
pdl">
+<script class="theme: Default; brush: java; gutter: false" type="syntaxhighlighter"><![CDATA[
 &lt;broker brokerName=&quot;broker&quot; ... &gt;
     ...
     &lt;persistenceAdapter&gt;
       &lt;levelDB directory=&quot;activemq-data&quot;/&gt;
@@ -112,18 +97,7 @@
     ...
   &lt;/broker&gt;
 ]]></script>
-</div></div>
-
-<h3 id="LevelDBStore-LevelDBProperties">LevelDB Properties</h3>
-
-<div class="table-wrap"><table class="confluenceTable"><tbody><tr><th
colspan="1" rowspan="1" class="confluenceTh"><p> property name </p></th><th
colspan="1" rowspan="1" class="confluenceTh"><p> default value </p></th><th
colspan="1" rowspan="1" class="confluenceTh"><p> Comments </p></th></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p> directory </p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p> "LevelDB" </p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p> The directory which the store will
use to hold it's data files. The store will create the directory if it does not already exist.
</p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p>
readThreads </p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>
10 </p></td><td colspan="1" rowspan="1" class="confluenceTd"><p> The
number of concurrent IO read threads to allowed. </p></td></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p> sync </p></td><td
colspan="1" rowspan=
 "1" class="confluenceTd"><p> true </p></td><td colspan="1" rowspan="1"
class="confluenceTd"><p> If set to false, then the store does not sync logging operations
to disk </p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p>
logSize </p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>
104857600 (100 MB) </p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>
The max size (in bytes) of each data log file before log file rotation occurs. </p></td></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p> logWriteBufferSize </p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p> 4194304 (4 MB) </p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p> That maximum amount of log data
to build up before writing to the file system. </p></td></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p> verifyChecksums </p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p> false </p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p> Se
 t to true to force checksum verification of all data that is read from the file system. </p></td></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p> paranoidChecks </p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p> false </p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p> Make the store error out as soon
as possible if it detects internal corruption. </p></td></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p> indexFactory </p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p> org.fusesource.leveldbjni.JniDBFactory,
org.iq80.leveldb.impl.Iq80DBFactory </p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>
The factory classes to use when creating the LevelDB indexes </p></td></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p> indexMaxOpenFiles </p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p> 1000 </p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p> Number of open files that can be
used by the index
 . </p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p>
indexBlockRestartInterval </p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>
16 </p></td><td colspan="1" rowspan="1" class="confluenceTd"><p> Number
keys between restart points for delta encoding of keys. </p></td></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p> indexWriteBufferSize </p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p> 6291456 (6 MB) </p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p> Amount of index data to build up
in memory before converting to a sorted on-disk file. </p></td></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p> indexBlockSize </p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p> 4096 (4 K) </p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p> The size of index data packed per
block. </p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p>
indexCacheSize </p></td><td colspan="1" rowspan="1" class
 ="confluenceTd"><p> 268435456 (256 MB) </p></td><td colspan="1" rowspan="1"
class="confluenceTd"><p> The maximum amount of off-heap memory to use to cache index
blocks. </p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p>
indexCompression </p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>
snappy </p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>
The type of compression to apply to the index blocks.  Can be snappy or none. </p></td></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p> logCompression </p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p> none </p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p> The type of compression to apply
to the log records. Can be snappy or none. </p></td></tr></tbody></table></div>
-
-
-<p>For tuning locking properties please take a look at <a shape="rect" href="pluggable-storage-lockers.html">Pluggable
storage lockers</a></p>
-
-<h2 id="LevelDBStore-AlsoSee">Also See</h2>
-
-<ul><li><a shape="rect" href="replicated-leveldb-store.html">Replicated
LevelDB Store</a> An extended version of this store which self replicates to other broker
nodes to increase message availability.</li></ul></div>
+</div></div><h3 id="LevelDBStore-LevelDBProperties">LevelDB Properties</h3><div
class="table-wrap"><table class="confluenceTable"><tbody><tr><th colspan="1"
rowspan="1" class="confluenceTh"><p>property name</p></th><th colspan="1"
rowspan="1" class="confluenceTh"><p>default value</p></th><th colspan="1"
rowspan="1" class="confluenceTh"><p>Comments</p></th></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p>directory</p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>"LevelDB"</p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>The directory which the store will
use to hold it's data files. The store will create the directory if it does not already exist.</p></td></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p>sync</p></td><td colspan="1"
rowspan="1" class="confluenceTd"><p>true</p></td><td colspan="1" rowspan="1"
class="confluenceTd"><p>If set to false, then the store does not sync logging operations
to disk</p></td></tr><tr><td colspan="1" row
 span="1" class="confluenceTd"><p>logSize</p></td><td colspan="1"
rowspan="1" class="confluenceTd"><p>104857600 (100 MB)</p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>The max size (in bytes) of each data
log file before log file rotation occurs.</p></td></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p>logWriteBufferSize</p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>4194304 (4 MB)</p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>That maximum amount of log data to
build up before writing to the file system.</p></td></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p>verifyChecksums</p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>false</p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>Set to true to force checksum verification
of all data that is read from the file system.</p></td></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p>paranoidChecks</p></td><td
colspan="1" rowspan="1" class="confluen
 ceTd"><p>false</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>Make
the store error out as soon as possible if it detects internal corruption.</p></td></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p>indexFactory</p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>org.fusesource.leveldbjni.JniDBFactory,
org.iq80.leveldb.impl.Iq80DBFactory</p></td><td colspan="1" rowspan="1" class="confluenceTd"><p>The
factory classes to use when creating the LevelDB indexes</p></td></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p>indexMaxOpenFiles</p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>1000</p></td><td colspan="1"
rowspan="1" class="confluenceTd"><p>Number of open files that can be used by the
index.</p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p>indexBlockRestartInterval</p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>16</p></td><td colspan="1"
rowspan="1" class="confluenceTd"><p>Number keys between r
 estart points for delta encoding of keys.</p></td></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p>indexWriteBufferSize</p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>6291456 (6 MB)</p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>Amount of index data to build up
in memory before converting to a sorted on-disk file.</p></td></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p>indexBlockSize</p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>4096 (4 K)</p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>The size of index data packed per
block.</p></td></tr><tr><td colspan="1" rowspan="1" class="confluenceTd"><p>indexCacheSize</p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>268435456 (256 MB)</p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>The maximum amount of off-heap memory
to use to cache index blocks.</p></td></tr><tr><td colspan="1"
rowspan="1" class="confluenceTd"><p>indexCompression</p></td><t
 d colspan="1" rowspan="1" class="confluenceTd"><p>snappy</p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>The type of compression to apply
to the index blocks. Can be snappy or none.</p></td></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p>logCompression</p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>none</p></td><td colspan="1"
rowspan="1" class="confluenceTd"><p>The type of compression to apply to the log records.
Can be snappy or none.</p></td></tr></tbody></table></div><p>For
tuning locking properties please take a look at <a shape="rect" href="pluggable-storage-lockers.html">Pluggable
storage lockers</a></p><h2 id="LevelDBStore-AlsoSee">Also See</h2><ul><li><a
shape="rect" href="replicated-leveldb-store.html">Replicated LevelDB Store</a> An
extended version of this store which self replicates to other broker nodes to increase message
availability.</li></ul></div>
         </td>
         <td valign="top">
           <div class="navigation">



Mime
View raw message