activemq-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From build...@apache.org
Subject svn commit: r936216 - in /websites/production/activemq/content: cache/main.pageCache pluggable-storage-lockers.html
Date Wed, 14 Jan 2015 22:21:49 GMT
Author: buildbot
Date: Wed Jan 14 22:21:49 2015
New Revision: 936216

Log:
Production update by buildbot for activemq

Modified:
    websites/production/activemq/content/cache/main.pageCache
    websites/production/activemq/content/pluggable-storage-lockers.html

Modified: websites/production/activemq/content/cache/main.pageCache
==============================================================================
Binary files - no diff available.

Modified: websites/production/activemq/content/pluggable-storage-lockers.html
==============================================================================
--- websites/production/activemq/content/pluggable-storage-lockers.html (original)
+++ websites/production/activemq/content/pluggable-storage-lockers.html Wed Jan 14 22:21:49
2015
@@ -82,7 +82,7 @@
   <tbody>
         <tr>
         <td valign="top" width="100%">
-<div class="wiki-content maincontent"><p>With 5.7.0 release, we introduced the
change on locking strategies for shared storage master/slave topologies. Previously storage
locking (and thus master election) was hard-coded directly in the particular store. So KahaDB
had only the option to use shared file lock, while JDBC was using database lock.</p><p>Now,
the storage locking is separated from the store, so you can implement your own locking strategies
if necessary (or tune existing ones). Of course, every store has it's own default locker.</p><h2
id="Pluggablestoragelockers-Lockers">Lockers</h2><p>Every locker must implement
the <a shape="rect" class="external-link" href="https://fisheye6.atlassian.com/browse/activemq/trunk/activemq-core/src/main/java/org/apache/activemq/broker/Locker.java?hb=true"
rel="nofollow">Locker</a> interface. So there are a couple of settings you can tune
for every locker:</p><div class="table-wrap"><table class="confluenceTable"><tbody><tr><th
colspan="1" r
 owspan="1" class="confluenceTh"><p>Property Name</p></th><th colspan="1"
rowspan="1" class="confluenceTh"><p>Default Value</p></th><th colspan="1"
rowspan="1" class="confluenceTh"><p>Description</p></th></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p><code>lockAcquireSleepInterval</code></p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>10000</p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>Delay interval in milliseconds between
lock acquire attempts</p></td></tr><tr><td colspan="1" rowspan="1"
class="confluenceTd"><p><code>failIfLocked</code></p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>false</p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>Should the start fail immediately
if lock cannot be obtained</p></td></tr></tbody></table></div><h2
id="Pluggablestoragelockers-PersistenceAdapters">Persistence Adapters</h2><p>Every
persistence adapter (or other broker service that wants to use locks) needs to implement the
<a shape="r
 ect" class="external-link" href="https://fisheye6.atlassian.com/browse/activemq/trunk/activemq-core/src/main/java/org/apache/activemq/broker/Lockable.java?r=1383400"
rel="nofollow">Lockable</a> interface. You can use a few settings for every persistence
adapter (that supports locking):</p><div class="table-wrap"><table class="confluenceTable"><tbody><tr><th
colspan="1" rowspan="1" class="confluenceTh"><p>Property Name</p></th><th
colspan="1" rowspan="1" class="confluenceTh"><p>Default Value</p></th><th
colspan="1" rowspan="1" class="confluenceTh"><p>Description</p></th></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p><code>useLock</code></p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>true</p></td><td colspan="1"
rowspan="1" class="confluenceTd"><p>can be used to turn off locking if necessary</p></td></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p><code>lockKeepAlivePeriod</code></p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>0</p></
 td><td colspan="1" rowspan="1" class="confluenceTd"><p>If bigger than 0, time
period (in milliseconds) to keep lock alive</p></td></tr></tbody></table></div><h2
id="Pluggablestoragelockers-ExistingLockers">Existing Lockers</h2><h3 id="Pluggablestoragelockers-SharedFileLocker">Shared
File Locker</h3><p>The Shared File Locker is the default locker for the KahaDB
persistence adapter. It locks a file to ensure that only the broker holding the lock (the
master) is granted access to the message store. You can configure it like this:</p><div
class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent
pdl">
+<div class="wiki-content maincontent"><p>With the 5.7.0 release of ActiveMQ the
storage locking mechanism used by a persistence adapter is pluggable. This feature only applies
to brokers configured in a shared storage master/slave topology. Prior to release 5.7.0 the
storage locking mechanism (and thus master election) was dictated by the choice of persistence
adapter. With the KahaDB persistence adapter, for example, the storage locking mechanism was
based on a shared file lock. Similarly, the JDBC persistence adapter used a database backed
storage lock.</p><p>Now that the choice of storage lock is divorced from that
of the persistence adapter one can mix and match. Storage locker pluggability is made possible
because all lockers must implement the <a shape="rect" class="external-link" href="https://fisheye6.atlassian.com/browse/activemq/trunk/activemq-broker/src/main/java/org/apache/activemq/broker/Locker.java?hb=true"
rel="nofollow">Locker</a> interface. This interface makes it e
 asy to implement your own storage locker when you have special requirements. Of course, every
persistence adapter still has its own default locker which works as before.</p><h2
id="Pluggablestoragelockers-Lockers">Lockers</h2><p>Every locker must implement
the <a shape="rect" class="external-link" href="https://fisheye6.atlassian.com/browse/activemq/trunk/activemq-broker/src/main/java/org/apache/activemq/broker/Locker.java?hb=true"
rel="nofollow">Locker</a> interface. The locker has the following configurable parameters:</p><div
class="table-wrap"><table class="confluenceTable"><tbody><tr><th colspan="1"
rowspan="1" class="confluenceTh"><p>Property Name</p></th><th colspan="1"
rowspan="1" class="confluenceTh"><p>Default Value</p></th><th colspan="1"
rowspan="1" class="confluenceTh"><p>Description</p></th></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p><code>lockAcquireSleepInterval</code></p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>10000</p></td><td
cols
 pan="1" rowspan="1" class="confluenceTd"><p>Delay interval in milliseconds between
lock acquire attempts</p></td></tr><tr><td colspan="1" rowspan="1"
class="confluenceTd"><p><code>failIfLocked</code></p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>false</p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>Should the start fail immediately
if lock cannot be obtained</p></td></tr></tbody></table></div><h2
id="Pluggablestoragelockers-PersistenceAdapters">Persistence Adapters</h2><p>Every
persistence adapter (or other broker service that wants to use locks) needs to implement the
<a shape="rect" class="external-link" href="https://fisheye6.atlassian.com/browse/activemq/trunk/activemq-broker/src/main/java/org/apache/activemq/broker/Lockable.java?r=1383400"
rel="nofollow">Lockable</a> interface. This adds the following configurable parameters:</p><div
class="table-wrap"><table class="confluenceTable"><tbody><tr><th colspan="1"
rowspan="1" class="confluenceTh"><p>Propert
 y Name</p></th><th colspan="1" rowspan="1" class="confluenceTh"><p>Default
Value</p></th><th colspan="1" rowspan="1" class="confluenceTh"><p>Description</p></th></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p><code>useLock</code></p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>true</p></td><td colspan="1"
rowspan="1" class="confluenceTd"><p>can be used to turn off locking if necessary</p></td></tr><tr><td
colspan="1" rowspan="1" class="confluenceTd"><p><code>lockKeepAlivePeriod</code></p></td><td
colspan="1" rowspan="1" class="confluenceTd"><p>0</p></td><td colspan="1"
rowspan="1" class="confluenceTd"><p>If bigger than 0, time period (in milliseconds)
to keep lock alive</p></td></tr></tbody></table></div><h2
id="Pluggablestoragelockers-ExistingLockers">Existing Lockers</h2><h3 id="Pluggablestoragelockers-SharedFileLocker">Shared
File Locker</h3><p>The Shared File Locker is the default locker for the KahaDB
persistence adapter. It locks a file to ensure that 
 only the broker holding the lock (the master) is granted access to the message store.</p><p>Example:</p><div
class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent
pdl">
 <script class="theme: Default; brush: xml; gutter: false" type="syntaxhighlighter"><![CDATA[&lt;persistenceAdapter&gt;
 	&lt;kahaDB directory = &quot;target/activemq-data&quot;&gt;
 		&lt;locker&gt;
@@ -90,7 +90,7 @@
 		&lt;/locker&gt;
 	&lt;/kahaDB&gt;
 &lt;/persistenceAdapter&gt;]]></script>
-</div></div><p>This locker implements the keepAlive method from 5.9.0 onwards
so there's no point in using lockKeepAlivePeriod settings on older versions than ActiveMQ
5.9.0. Note that as of ActiveMQ 5.9.0 the KahaDB persistence adapter can also use the Lease
Database Locker (see below).</p><h3 id="Pluggablestoragelockers-DatabaseLocker">Database
Locker</h3><p>The Database Locker is the default locker for the JDBC persistence
adapter. It locks a database table in a transaction to ensure that only single resource is
used. You can configure it like this:</p><div class="code panel pdl" style="border-width:
1px;"><div class="codeContent panelContent pdl">
+</div></div><p>This locker implements the <code>keepAlive</code>
method from 5.9.0 onwards so there's no point in using <code>lockKeepAlivePeriod</code>
settings on older versions than ActiveMQ 5.9.0. Note that as of ActiveMQ 5.9.0 the KahaDB
persistence adapter can also use the Lease Database Locker (see below).</p><h3 id="Pluggablestoragelockers-DatabaseLocker">Database
Locker</h3><p>The Database Locker is the default locker for the JDBC persistence
adapter. It locks a database table in a transaction to ensure that only single resource is
used.</p><p>Example:</p><div class="code panel pdl" style="border-width:
1px;"><div class="codeContent panelContent pdl">
 <script class="theme: Default; brush: xml; gutter: false" type="syntaxhighlighter"><![CDATA[&lt;persistenceAdapter&gt;
     &lt;jdbcPersistenceAdapter dataDirectory=&quot;${activemq.data}&quot; dataSource=&quot;#mysql-ds&quot;
lockKeepAlivePeriod=&quot;10000&quot;&gt;
         &lt;locker&gt;
@@ -98,7 +98,7 @@
         &lt;/locker&gt;
     &lt;/jdbcPersistenceAdapter&gt;
 &lt;/persistenceAdapter&gt;]]></script>
-</div></div><p>The Database Locker uses its&#160;<code>keepAlive</code>
method to ensure the broker still holds the lock. You can set the keep alive period using
the <code>lockKeepAlivePeriod</code> property. The default period is 30000 ms.
If a broker fails to acquire the lock on the database, it will retry every lockAcquireSleepInterval
milliseconds.</p><p>This locker opens a JDBC transaction against a database table
(activemq_lock) that lasts as long as the broker remains alive. This locks the entire table
and prevents another broker from accessing the store. In most cases this will be a fairly
long running JDBC transaction which occupies resources on the database over time.</p><p>A
problem with this locker can occur when the master broker crashes or loses its connection
to the database. The information about the lock remains in the database, until the database
responds to the half closed socket connection via a tcp timeout. The database lock expiry
requirement can prevent the sl
 ave from starting for a period. In addition, if the database supports failover, and the connection
is dropped in the event of a replica failover, that JDBC transaction will be rolled back.
The broker sees this as a failure and both master and slave will again compete for a lock.</p><h3
id="Pluggablestoragelockers-LeaseDatabaseLocker">Lease Database Locker</h3><p>The
Lease Database Locker solves the master/slave problems of the default Database Locker. It
does not open a long running JDBC transaction but lets the master broker acquire a lock that's
only valid for a short period. To retain the lock the master broker must periodically extend
the lock's lease. The slave broker also checks periodically to see if the lease has expired.
The lease can survive a db replica failover. You can configure it like this:</p><div
class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent
pdl">
+</div></div><p>The Database Locker uses its&#160;<code>keepAlive</code>
method to ensure the broker still holds the lock. You can set the keep alive period using
the <code>lockKeepAlivePeriod</code> property. The default period is 30000 ms.
If a broker fails to acquire the lock on the database, it will retry every <code>lockAcquireSleepInterval</code>
milliseconds.</p><p>This locker opens a JDBC transaction against a database table
(<code>activemq_lock</code>) that lasts as long as the broker remains alive. This
locks the entire table and prevents another broker from accessing the store. In most cases
this will be a fairly long running JDBC transaction which occupies resources on the database
over time.</p><p>A problem with this locker can arise when the master broker crashes
or loses its connection to the database causing the lock to remain in the database until the
database responds to the half closed socket connection via a TCP timeout. The database lock
expiry requirement can pr
 event the slave from starting some time. In addition, if the database supports failover,
and the connection is dropped in the event of a replica failover, that JDBC transaction will
be rolled back. The broker sees this as a failure. Both master and slave brokes will again
compete for a lock.</p><h3 id="Pluggablestoragelockers-LeaseDatabaseLocker">Lease
Database Locker</h3><p>The Lease Database Locker was created to solve the shortcomings
of the Database Locker. The Lease Database Locker does not open a long running JDBC transaction.
Instead it lets the master broker acquire a lock that's valid for a fixed (usually short)
duration after which it expires. To retain the lock the master broker must periodically extend
the lock's lease before it expires. Simultaneously the slave broker also checks periodically
to see if the lease has expired. If, for whatever reason, the master broker fails to update
its lease on the lock the slave will take ownership of the lock becoming the new master 
 in the process. The leased lock can survive a DB replica failover.</p><p>Example:</p><div
class="code panel pdl" style="border-width: 1px;"><div class="codeContent panelContent
pdl">
 <script class="theme: Default; brush: xml; gutter: false" type="syntaxhighlighter"><![CDATA[&lt;persistenceAdapter&gt;
 	&lt;jdbcPersistenceAdapter dataDirectory=&quot;${activemq.data}&quot; dataSource=&quot;#mysql-ds&quot;
lockKeepAlivePeriod=&quot;5000&quot;&gt;
 		&lt;locker&gt;
@@ -106,10 +106,10 @@
 		&lt;/locker&gt;
 	&lt;/jdbcPersistenceAdapter&gt;
 &lt;/persistenceAdapter&gt;]]></script>
-</div></div><p>In order for this mechanism to work correctly, each broker
in the master/slave pair must have a different <code>brokerName</code> attribute
defined on the <code>broker</code> tag or use the&#160;<code><span
style="line-height: 1.4285715;">lease-database-locker </span></code><span
style="line-height: 1.4285715;"><code>leaseHolderId</code> attribute, as it
is this value that is used to reserve a lease.</span></p><p>The lease based
lock is acquired by blocking at startup. It is then retained for a period whose duration (in
ms) is given by the <code>lockKeepAlivePeriod</code> attribute. To retain the
lock the master broker periodically extends its lease by <code>lockAcquireSleepInterval</code>
milliseconds each time. In theory, therefore, the master broker is always (<code>lockAcquireSleepInterval</code>&#160;<code>-</code>
<code>lockKeepAlivePeriod</code>) ahead of the slave broker with regard to the
lease. It is imperative that <code>lockAcquireSleepInterval &gt; lockKe
 epAlivePeriod</code>, to ensure the lease is always current. As of ActiveMQ 5.9.0 a
warning message is logged if this condition is not met.</p><p>In the simplest
case, the clocks between master and slave must be in sync for this solution to work properly.
If the clocks cannot be in sync, the locker can use the system time from the database CURRENT
TIME and adjust the timeouts in accordance with their local variance from the db system time.
If&#160;<code>maxAllowableDiffFromDBTime</code> is greater than zero the local
periods will be adjusted by any delta that exceeds <code>maxAllowableDiffFromDBTime</code>.</p>
   <div class="aui-message hint shadowed information-macro">
+</div></div><p>In order for this mechanism to work correctly, each broker
in the master/slave pair must have a different <code>brokerName</code> attribute
defined on the <code>broker</code> tag or use the <span><code>leaseHolderId</code>
attribute</span> on the&#160;<code><span style="line-height: 1.4285715;">lease-database-locker</span></code><span>,
as it is this value that is used to reserve a lease.</span></p><p>The lease
based lock is acquired by blocking at startup. It is then retained for a period whose duration
(in ms) is given by the <code>lockKeepAlivePeriod</code> attribute. To retain
the lock the master broker periodically extends its lease by <code>lockAcquireSleepInterval</code>
milliseconds each time. In theory, therefore, the master broker is always (<code>lockAcquireSleepInterval</code>&#160;<code>-</code>
<code>lockKeepAlivePeriod</code>) ahead of the slave broker with regard to the
lease. It is imperative that <code>lockAcquireSleepInterval &gt; lockKeepAlivePerio
 d</code>, to ensure the lease is always current. As of ActiveMQ 5.9.0 a warning message
is logged if this condition is not met.</p><p>In the simplest case, the clocks
between master and slave must be in sync for this solution to work properly. If the clocks
cannot be in sync, the locker can use the system time from the database CURRENT TIME and adjust
the timeouts in accordance with their local variance from the DB system time. If&#160;<code>maxAllowableDiffFromDBTime</code>
is greater than zero the local periods will be adjusted by any delta that exceeds <code>maxAllowableDiffFromDBTime</code>.</p>
   <div class="aui-message hint shadowed information-macro">
                             <span class="aui-icon icon-hint">Icon</span>
                 <div class="message-content">
-                            <p>It is important to know if the default rules your JDBC
driver uses for converting TIME values are JDBC compliant. If you're using MySQL, for example,
the driver's JDBC URL should contain&#160;<code>useJDBCCompliantTimezoneShift=true</code>
to ensure that TIME value conversion is JDBC compliant. If not the locker could report a large
time difference when it compares the retrieved lease expiration time against the current system
time. Consult your JDBC driver's manual for more details.</p>
+                            <p>It is important to know if the default rules your JDBC
driver uses for converting <code>TIME</code> values are JDBC compliant. If you're
using MySQL, for example, the driver's JDBC URL should contain&#160;<code>useJDBCCompliantTimezoneShift=true</code>
to ensure that <code>TIME</code> value conversion is JDBC compliant. If not the
locker could report a large time difference when it compares the retrieved lease expiration
time against the current system time. Consult your JDBC driver's documentation for more details.</p>
                     </div>
     </div>
 <p>As of ActiveMQ 5.9.0 the lease database locker can be used in conjunction with the
KahaDB persistence adapter. However, this particular combination requires that the lease database
locker element contains a <code>&lt;statements/&gt;</code> child element.
In the example below the&#160;<code>lockTableName</code> is also configured,
although doing so is not mandatory.</p><div class="code panel pdl" style="border-width:
1px;"><div class="codeContent panelContent pdl">
@@ -127,7 +127,7 @@
 		&lt;/locker&gt;
 	&lt;/kahaDB&gt;
 &lt;/persistenceAdapter&gt;]]></script>
-</div></div><p>To see the complete list of attributes and SQL statements
that can be overridden see the <a shape="rect" class="external-link" href="https://fisheye6.atlassian.com/browse/activemq/trunk/activemq-jdbc-store/src/main/java/org/apache/activemq/store/jdbc/Statements.java?hb=true"
rel="nofollow">Statements</a> class.</p><p>&#160;</p><p>&#160;</p></div>
+</div></div><p>To see the complete list of attributes and SQL statements
that can be overridden see the <a shape="rect" class="external-link" href="https://fisheye6.atlassian.com/browse/activemq/trunk/activemq-jdbc-store/src/main/java/org/apache/activemq/store/jdbc/Statements.java?hb=true"
rel="nofollow">Statements</a> class. When the KahaDB persistence adapter is configured
to use the <code>lease-database-locker</code> you must configure the broker to
use your own IO exception handler as neither the <code>DefaultIOExceptionHandler</code>
nor the <code>JDBCIOExceptionHandler</code> will work correctly with this combination.
See <a shape="rect" href="configurable-ioexception-handling.html">Configurable IOException
Handlers</a> for details on how to write a handler.</p><p>As of ActiveMQ
5.11, however, the <code>JDBCIOExceptionHandler</code> has been deprecated. It
has been replaced by the <code>org.apache.activemq.util.LeaseLockerIOExceptionHandler</code>
which will work with any pers
 istence adapter that supports pluggable storage lockers.</p><p>&#160;</p><p>&#160;</p></div>
         </td>
         <td valign="top">
           <div class="navigation">



Mime
View raw message