activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gary Tully (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (AMQ-4556) Store based cursor "forgets" messages
Date Mon, 13 Jan 2014 20:36:52 GMT

    [ https://issues.apache.org/jira/browse/AMQ-4556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13869941#comment-13869941
] 

Gary Tully commented on AMQ-4556:
---------------------------------

and if you get something that reproduces, please validate against a 5.10-SNAPSHOT to see if
the issue exists on trunk. thx

> Store based cursor "forgets" messages
> -------------------------------------
>
>                 Key: AMQ-4556
>                 URL: https://issues.apache.org/jira/browse/AMQ-4556
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Message Store
>    Affects Versions: 5.7.0, 5.8.0
>         Environment: Durable queues with JDBC persistent store in a master/slave configuration
with NIO transport
>            Reporter: SunGard Global Services Germany
>            Priority: Critical
>         Attachments: ActiveMQClient.java, ActiveMQClient2.java, activemq.xml
>
>
> This issue seems to relate to AMQ-2009 and the referenced articles there.
> h3. The issue:
> After a give period of time, the broker seems to "forget" about received messages on
a queue.
> Receiving of new messages and delivering them is unaffected. Just some are lost in the
store.
> Checking the DB tables, the missing messages do exists there.
> ActiveMQ statistics counters are also aware of them and count those messages as "pending"
in the ActiveMQ console (first colummn). But open the details of a queue in ActiveMQ console
do not show them at all.
> h3. Analysis so far
> We tried several settings for the queues which had no impact on the issue:
> * Activating/Deactivating useCache
> * Setting different prefetch values from 0, 1, 100 (at least 1 seems to relax the issue
a little bit)
> * KahaDB and JDBC persistent (Oracle and MSSQL(jTDS)) are affected
> Also some characteristics about affected/not affected queues might help to analyse this
further:
> * We have one queue with just one producer and many consumers whereas the message is
quite small (just headers): No problem on that queue even after thousend of messages
> * Queues with multiple producers and consumers and payloaded text-messages seem to be
unaffected - Maybe the JDBC persistent store trottels the processing enough to "solve" the
issue
> * Queues with multiple producers and consumers and small messages (just headers) seem
to "enable" the issue. Even with few messages the issue appears
> h3. "Recovering" lost messages
> Shutdown the (master) broker and restart it. With failover transport this happens transparent
for the clients.
> On master startup, all messages from the store (DB) are scanned and "rediscovered"
> h3. Workaround
> Use another cursor - VM seems to be fine. Additional settings might be required to handle
shotcommings of the VM cursor (max memory, queue memory etc.)
> {code:xml}
>                 <policyEntry queue=">" ...>
>                   <pendingQueuePolicy>
>                     <vmQueueCursor/>
>                   </pendingQueuePolicy>
>                 </policyEntry>
> {code}
> h3. Test code
> I was not able to create a self-contained unit test.
> But the attached code can be used to reproduce the issue quite reliable.
> It sends 3000 messages. WIth the following settings it will fail/will work correctly:
> || Message Size || ActiveMQ cursor || Result ||
> | 1 MB (leave line 20 as it is) | vmQueueCursor | (/) - All messages delivered|
> | 1 MB (leave line 20 as it is) | store based Cursor| (/) - All messages delivered|
> | remove comment from line 20 | vmQueueCursor| (/) - All messages delivered|
> | remove comment from line 20 | store based Cursor| (x) - Some (~0,5% messages lost)|
> For completness also the used activemq-broker.xml was attached - This is the default
one with the JDBC persistent store added.



--
This message was sent by Atlassian JIRA
(v6.1.5#6160)

Mime
View raw message