activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "SunGard Global Services Germany (JIRA)" <>
Subject [jira] [Updated] (AMQ-4556) Store based cursor "forgets" messages
Date Sun, 26 May 2013 14:07:24 GMT


SunGard Global Services Germany updated AMQ-4556:

    Attachment: activemq.xml
> Store based cursor "forgets" messages
> -------------------------------------
>                 Key: AMQ-4556
>                 URL:
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Message Store
>    Affects Versions: 5.7.0, 5.8.0
>         Environment: Durable queues with JDBC persistent store in a master/slave configuration
with NIO transport
>            Reporter: SunGard Global Services Germany
>            Priority: Critical
>         Attachments:, activemq.xml
> This issue seems to relate to AMQ-2009 and the referenced articles there.
> h3. The issue:
> After a give period of time, the broker seems to "forget" about received messages on
a queue.
> Receiving of new messages and delivering them is unaffected. Just some are lost in the
> Checking the DB tables, the missing messages do exists there.
> ActiveMQ statistics counters are also aware of them and count those messages as "pending"
in the ActiveMQ console (first colummn). But open the details of a queue in ActiveMQ console
do not show them at all.
> h3. Analysis so far
> We tried several settings for the queues which had no impact on the issue:
> * Activating/Deactivating useCache
> * Setting different prefetch values from 0, 1, 100 (at least 1 seems to relax the issue
a little bit)
> * KahaDB and JDBC persistent (Oracle and MSSQL(jTDS)) are affected
> Also some characteristics about affected/not affected queues might help to analyse this
> * We have one queue with just one producer and many consumers whereas the message is
quite small (just headers): No problem on that queue even after thousend of messages
> * Queues with multiple producers and consumers and payloaded text-messages seem to be
unaffected - Maybe the JDBC persistent store trottels the processing enough to "solve" the
> * Queues with multiple producers and consumers and small messages (just headers) seem
to "enable" the issue. Even with few messages the issue appears
> h3. "Recovering" lost messages
> Shutdown the (master) broker and restart it. With failover transport this happens transparent
for the clients.
> On master startup, all messages from the store (DB) are scanned and "rediscovered"
> h3. Workaround
> Use another cursor - VM seems to be fine. Additional settings might be required to handle
shotcommings of the VM cursor (max memory, queue memory etc.)
> {code:xml}
>                 <policyEntry queue=">" ...>
>                   <pendingQueuePolicy>
>                     <vmQueueCursor/>
>                   </pendingQueuePolicy>
>                 </policyEntry>
> {code}
> h3. Test code
> I was not able to create a self-contained unit test.
> But the attached code can be used to reproduce the issue quite reliable.
> It sends 3000 messages. WIth the following settings it will fail/will work correctly:
> || Message Size || ActiveMQ cursor || Result ||
> | 1 MB (leave line 20 as it is) | vmQueueCursor | (/) |
> | 1 MB (leave line 20 as it is) | store based Cursor| (/) |
> | remove comment from line 20 | vmQueueCursor| (/) |
> | remove comment from line 20 | store based Cursor| (x) |
> For completness also the used activemq-broker.xml was attached - This is the default
one with the JDBC persistent store added.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see:

View raw message