activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gary Tully (JIRA)" <j...@apache.org>
Subject [jira] Updated: (AMQ-2573) Failover with async rollback of message audit can lead to over eager duplicate suppression (auto acking) and apparent missing messages
Date Wed, 20 Jan 2010 16:56:44 GMT

     [ https://issues.apache.org/activemq/browse/AMQ-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Gary Tully updated AMQ-2573:
----------------------------

    Description: 
with multiple unordered message consumers on a connection and failover with default prefetch
recovery processing can lead to incorrect duplicate detection.
Duplicates are expected as all unacked messages will be redelivered on recovery. An active
consumer may have unconsumed messages which need to be rolled back in the message audit. currently
this rollback is occurring at an indeterminate time w.r.t to future dispatch.
given consumer 1 has unconsumed messages 0-10, consumer 2 11-20 etc.. all need to be rolled
back prior to recovery of consumer 3 which may get any message in the range 0 - 20 delivered
to it. Currently, rollbacks are done on the next delivery. 
All audit rollbacks need to be complete before any new delivery occurs on a recovered connection.

The result of over eager duplicate suppression is missing messages as the duplicates are auto
acked.
Symptom: missing messages from a consumer perspective. With debug logging on {code}log4j.logger.org.apache.activemq.ActiveMQMessageConsumer=DEBUG{code}
it is possible to catch the duplicate suppression. If your missing messages are in this set,
you may have this problem.

workaround is to reduce the prefetch or even use a pull consumer, prefetch=0 such that unconsumed
messages do not back up in the consumers dispatch queue pending receive() calls. With async
listeners where the consumers can keep up with dispatch, this will not be a problem,

  was:
with multiple unordered message consumers on a connection and failover with default prefetch
recovery processing can lead to incorrect duplicate detection.
Duplicates are expected as all unacked messages will be redelivered on recovery. An active
consumer may have unconsumed messages which need to be rolled back in the message audit. currently
this rollback is occurring at an indeterminate time w.r.t to future dispatch.
given consumer 1 has unconsumed messages 0-10, consumer 2 11-20 etc.. all need to be rolled
back prior to recovery of consumer 3 which may get an message in the range 0 - 20 delivered
to it. Currently, rollbacks are done on the next delivery. 
All audit rollbacks need to be complete before any new delivery occurs on a recovered connection.

The result of over eager duplicate suppression is missing messages as the duplicates are auto
acked.
Symptom: with debug logging on {code}log4j.logger.org.apache.activemq.ActiveMQMessageConsumer=DEBUG{code}
it is possible to catch the duplicate suppression. If your missing messages are in this set,
you may have this problem.


> Failover with async rollback of message audit can lead to over eager duplicate suppression
(auto acking) and apparent missing messages
> --------------------------------------------------------------------------------------------------------------------------------------
>
>                 Key: AMQ-2573
>                 URL: https://issues.apache.org/activemq/browse/AMQ-2573
>             Project: ActiveMQ
>          Issue Type: Bug
>          Components: Broker
>    Affects Versions: 5.3.0
>            Reporter: Gary Tully
>            Assignee: Gary Tully
>             Fix For: 5.3.1, 5.4.0
>
>
> with multiple unordered message consumers on a connection and failover with default prefetch
recovery processing can lead to incorrect duplicate detection.
> Duplicates are expected as all unacked messages will be redelivered on recovery. An active
consumer may have unconsumed messages which need to be rolled back in the message audit. currently
this rollback is occurring at an indeterminate time w.r.t to future dispatch.
> given consumer 1 has unconsumed messages 0-10, consumer 2 11-20 etc.. all need to be
rolled back prior to recovery of consumer 3 which may get any message in the range 0 - 20
delivered to it. Currently, rollbacks are done on the next delivery. 
> All audit rollbacks need to be complete before any new delivery occurs on a recovered
connection.
> The result of over eager duplicate suppression is missing messages as the duplicates
are auto acked.
> Symptom: missing messages from a consumer perspective. With debug logging on {code}log4j.logger.org.apache.activemq.ActiveMQMessageConsumer=DEBUG{code}
it is possible to catch the duplicate suppression. If your missing messages are in this set,
you may have this problem.
> workaround is to reduce the prefetch or even use a pull consumer, prefetch=0 such that
unconsumed messages do not back up in the consumers dispatch queue pending receive() calls.
With async listeners where the consumers can keep up with dispatch, this will not be a problem,

-- 
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.


Mime
View raw message