activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jeff Wartes <jwar...@whitepages.com>
Subject Coping with forgetful subscribers
Date Thu, 29 Jul 2010 01:15:17 GMT

Given a queue and the assumption that messages are idempotent, I'm looking for a way to have
a given consumer go 'back in time', and re-play some period or even all available messages
prior to resuming normal consumption. Possibly as much as the last 24 hours worth.
The trick is that the consumer may have already consumed some of these messages, but forgotten
that it had done so past a certain point in time. (This is by design, and I can get into the
reasons, but that's going to be a longer post)

We can assume that these bouts of amnesia coincide with the consumer disconnecting and reconnecting
again, so something like subscription recovery<http://activemq.apache.org/subscription-recovery-policy.html>
looks close, but this seems to be intended for cases where:

*         We're not going very far back in time. (does this feature store messages in something
other than memory if it's a durable queue?)

*         The consumer never actually got the messages that the broker is re-sending. (Does
the broker blindly re-send, or does it only send messages it thinks the consumer missed?)

Another thought was to keep a "historical queue" of messages that never has any subscribers,
but receives a copy of every message with a durability of 24 hours. The idea would be that
between the amnesiac disconnect and reconnect, we insert some steps, such that:

1.       The subscriber stops listening to its normal queue (disconnect)

2.       We forget that we got the last six hours of messages (amnesia strikes)

3.       We clone the "historical queue" including all messages

4.       We consume all messages from the cloned queue (get back to where we were)

5.       The subscriber resumes listening to its normal queue (reconnect)
This would allow unlimited replay of messages while still consuming in the normal sense. However,
I have not come across documentation that allows a client to request a queue be "cloned",
complete with unconsumed messages.

Is this possible with ActiveMQ, or do I need to find another solution for message-replay?


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message