activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Strachan" <james.strac...@gmail.com>
Subject Re: Failover topic subscribers
Date Thu, 06 Jul 2006 16:35:34 GMT
On 7/6/06, bmadigan <bmadigan@orbitz.com> wrote:
> Would it be possible to implement this as a Broker plugin or would more
> fundamental changes need to be made? I don't think I have enough control
> over sending messages by overriding send() and acknowledge() in
> BrokerFilter.

Very efficient version
================
Its a reasonable amount of work to do very efficiently (thats often
the way - its often easy to do things, but much harder to do them very
efficiently :).

Basically for a really efficient implementation we need to create a
version of durable topics which supports out of order acknowledgements
(so rather than keeping the last ack'd message ID per subscription
we'd maintain all the to-be-ack'd message Ids) then we'd need to build
a Queue abstraction around an individual topic subscription - then use
a BrokerFilter to delegate to the new types of durable topics if they
are 'queue enabled' etc.

All of this would be required to allow the message to be persisted
once, then each consumer just has a small acknowledgement table etc.
So a fair amount of work to do really efficiently.

However given the usefulness of such a feature - how about we
implement a much simpler solution to get the feature done sooner...


Simpler to implement version
=======================
A simpler solution could just be that we reuse the existing queue
implementations (so each 'virtual topic subscription' actually gets
its own physical queue) and we in effect just copy the message to all
the physical queues using one actual queue per logical topic
subscription. This would be fairly simple to implement using an
inteceptor - and is fairly simple to do in fact - the only downside is
it results in the message being persisted N times, once for each
subscription.

Basically we could do something like this...

* we configure one or more wildcards to indicate the 'virtual topics'
to use.  e.g. something like this in the activemq.xml...

<virtualTopic topic="FOO.>" queuePrefix="VirtualTopic."/>

* when a durable topic message is sent to a virtual topic FOO.BAR  we
dispatch it to the non-persistent Topic region as before, but instead
of using the persistent topic region we do the following...

* look for all the queues called VirtualTopic.*.FOO.BAR and send a
copy of the message to it. (We could maybe add some prefix to the
beginning to separate out the physical implementation queues for the
virtual topics).

* then any consumer can subscribe to a queue called
VirtualTopic.${subscriberName}.FOO.BAR which if you are the first
person to use a given subscriptionName the effect is to dynamically
create a virtual durable topic subscription for ${subscriptionName}.
You would then be using regular queue semantics - so folks can browse
the queue, delete messages from the queue and so forth.

e.g. we could create 100 consumers on VirtualTopic.A.FOO.BAR and 50
consumers on VirtualTopic.B.FOO.BAR which would logically represent 2
virtual topic subscriptions of topic FOO.BAR. i.e. you'd then get all
the benefit of queues - load balancing, parallelization, failover
along with features like exclusive queues and message groups - yet the
producer is a regular topic publisher and the consumer sides are
queues - so the effect is all standard JMS contracts.

Incidentally the reason I'm suggesting to use queue names of the form
${prefix}.${subscriberName}.${topicName} is so that we could then
support wildcards on the topics.

e.g. VirtualTopic.consumerAbc.Products.Electonics.>


I'd really love this feature - as durable topics in JMS are nowhere
near as useful as they should be - so starting with the simpler
approach sounds good to me :)

Anyone fancy taking a stab at implementing it? :)

-- 

James
-------
http://radio.weblogs.com/0112098/

Mime
View raw message