activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Tully <gary.tu...@gmail.com>
Subject Re: Coping with forgetful subscribers
Date Thu, 29 Jul 2010 10:01:11 GMT
Have a read of http://activemq.apache.org/virtual-destinations.html
I can imagine a variant of a filtered destination that sets an message
expiry duration such that it maintains a time limited mirror of a
queue. Then a composite destination that also pulls from the mirror.
If you implement your own destination interceptor you can have
complete control of where messages end up w.r.t to destinations.

I also wonder about ack modes, could the amnesia include forgetting to
ack or commit, that seems easier as all of the received but un acked
messages will still be available for redelivery once the consumer
disconnects.

On 29 July 2010 02:15, Jeff Wartes <jwartes@whitepages.com> wrote:
>
> Given a queue and the assumption that messages are idempotent, I'm looking for a way
to have a given consumer go 'back in time', and re-play some period or even all available
messages prior to resuming normal consumption. Possibly as much as the last 24 hours worth.
> The trick is that the consumer may have already consumed some of these messages, but
forgotten that it had done so past a certain point in time. (This is by design, and I can
get into the reasons, but that's going to be a longer post)
>
> We can assume that these bouts of amnesia coincide with the consumer disconnecting and
reconnecting again, so something like subscription recovery<http://activemq.apache.org/subscription-recovery-policy.html>
looks close, but this seems to be intended for cases where:
>
> *         We're not going very far back in time. (does this feature store messages
in something other than memory if it's a durable queue?)
>
> *         The consumer never actually got the messages that the broker is re-sending.
(Does the broker blindly re-send, or does it only send messages it thinks the consumer missed?)
>
> Another thought was to keep a "historical queue" of messages that never has any subscribers,
but receives a copy of every message with a durability of 24 hours. The idea would be that
between the amnesiac disconnect and reconnect, we insert some steps, such that:
>
> 1.       The subscriber stops listening to its normal queue (disconnect)
>
> 2.       We forget that we got the last six hours of messages (amnesia strikes)
>
> 3.       We clone the "historical queue" including all messages
>
> 4.       We consume all messages from the cloned queue (get back to where we were)
>
> 5.       The subscriber resumes listening to its normal queue (reconnect)
> This would allow unlimited replay of messages while still consuming in the normal sense.
However, I have not come across documentation that allows a client to request a queue be "cloned",
complete with unconsumed messages.
>
> Is this possible with ActiveMQ, or do I need to find another solution for message-replay?
>
>



-- 
http://blog.garytully.com

Open Source Integration
http://fusesource.com

Mime
View raw message