activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Nicholson <robert.nichol...@gmail.com>
Subject 1->N queue model for high volume traffic?
Date Wed, 23 Sep 2009 05:40:52 GMT
Is there ever an advantage to doing the following when order of  
processing matters?

There is a need to preserve order based on some characteristic of the  
message.

Therefore multiple consumers from the queue without the use of  
selectors is not an appropriate solution.

Instead, single consumer of the queue but this consumer will hash/ 
dispatch based on the characteristic
of the message to other queues and those queues have concurrent  
listeners one for each queue. (order of processing within the queue is  
preserved)

In my tests that seems to be quicker than single consumer only based  
on approaches where I'm required to use
the database to keep track of which messages I've already processed  
and then acknowledge the message
explicitly after processing. If there's any outage my database working  
table reflects what is left to do upon
restart of the process. The downside is that there's significant  
overhead in maintaining this table.

In the above example the idea is that when you use a transacted  
session if there's an interruption in processing
that causes a rollback of the session then redelivery of those  
messages that were not processed will occur. Therefore it's
no longer necessary to maintain this working table.

With this approach you commit the session in the same underlying  
thread that is responsible for delivery
to your listener.

....

Which DMLC (Spring) scenario allows you to process messages off the  
queue but still be able to
know that if there's an interruption that redelivery of all  
unprocessed messages is guaranteed to occur?
Obviously anything that acknowledges the message before processing  
start is not a viable solution.



Mime
View raw message