Mailing-List: contact activemq-users-help@geronimo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: activemq-users@geronimo.apache.org
Received-SPF: pass (herse.apache.org: domain of chase@bullhorn.com designates
 63.246.10.50 as permitted sender)
Message-ID: <5769805.1165588833274.JavaMail.cfservice@webserver6>
From: Chase Seibert <chase@bullhorn.com>
To: activemq-users@geronimo.apache.org
Subject: RE: uncommitted transactions preventing queue processing
Mime-Version: 1.0
Content-Type: multipart/mixed;
	boundary="----=_Part_6044_17819061.1165588833274"
Date: 8 Dec 2006 09:40:33 -0500

------=_Part_6044_17819061.1165588833274
Content-Type: multipart/alternative;
	boundary="----=_Part_6045_11665634.1165588833274"

------=_Part_6045_11665634.1165588833274
Content-Type: text/plain; charset="ISO-8859-1"
Content-Transfer-Encoding: 7bit


The suggestion I'm leaning twoards is to simply spawn many consumer threads so that if some go into a "waiting" state for a redelivery, the over-all queue will keep processing. However, I see some problems with this model that I'm not sure how to solve. Namely, I'm not sure how to create multiple consumers with sessionTransacted = true.

 
Perhaps this is a Spring issue and not an ActiveMQ one. My testing so far shows that the DefaultMessageListenerContainer settings sessionTransacted = true and concurrentConsumers > 1 are somewhat mutually exclusive. It seems that when the session is transacted, there is effectively only one consumer. Other consumer threads are spawned, but only one is allowed to enter the onMessage() block at once. Maybe the confusion is on my part; it seems that sessionTransacted is treating the entire queue as transacted, when my real goal is to have the individual messages be transacted. 

 
If I could implement transacted messages with a configurable number of consumers, that would help somewhat. But I would still be worried about a high volume of redelivery messages choking out ones that would be successful on the first run. I could easily fill 100 consumer threads with redelivery attempts when there are thousands of messages queued that could be delivered on the first shot. My initial design was to do redelivery on 1/5/15 minute cycles, which would make this even more likely.

 
If I cannot use sessionTransacted, then I will have to fall back to untransacted messages, at which point I can't use a redelivery policy at all (they only go into effect on a rollback). I would implement a queue for each redelivery attempt (maybe 2 or 3 of them), and simply put a Thread.Sleep(x) as the first step in each onMessage() processor to do the redelivery delay.

 
The deeper I go down that path, the more it seems like ActiveMQ is not the right tool for the job. Either that, or I'm not understanding something; which is entirely possible. 

 
   -Chase


Application Developer

B U L L H O R N


Staffing and Recruiting Software On Demand 

The Last Software You'll Ever Need

33-41 Farnsworth Street, 5th Floor, Boston, MA 02210
617.478.9119 

www.bullhorn.com


-----Original Message-----
From:James Strachan james.strachan@gmail.com 
To: "activemq-users@geronimo.apache.org" <activemq-users@geronimo.apache.org>;
Sent: Dec 8, 2006 05:03:44 AM
Subject: Re: uncommitted transactions preventing queue processing

On 7 Dec 2006 16:52:42 -0500, Chase Seibert wrote: 
> 
> *** Caveat: We are in the process of implementing our first ActiveMQ project, and we may be missing some basic configuration. *** 
> 
> 
> Our research shows that with ActiveMQ, transacted messages queued after the currently processing message will not get dequeued until that first transaction is committed. 

Yes. The current message must be processed & replayed until its replay 
counter is reached on which time its then failed back to the broker 

> This remains true even if the current transacted message has failed once and is pending redelivery. No other messages are processed until the first redelivery is successful, or the maximumRedeliveries threshold is reached. 

Yes. If this were not the case then queue ordering would be broken as 
soon as a redelivery occurs. 


> Our scenario involves transacted messages that do not necessarily need to be processed in the order they were inserted onto the queue. In other words, if a message listener throws an exception on a particular message, we still want the messages behind that one to get processed immediately. 

Either set a prefetch value of 1 (so no consumer gets more than one 
message at once) or set the maximum redeliveries to 1 to avoid a bad 
message blocking the next one behind. But if you don't care about 
order - why not just run lots of consumers so its not a big deal if 
one consumer replays a bad message a few times? 


> The explanation on that issue was that ActiveMQ preserves the order of the queue. In this case, subsequent messages cannot be processed until a previous one is committed in order to maintain the message order. In contacting the poster of that issue, the resolution was to use a secondary queue to reprocess failures. 

So a message that fails to be processed correctly is replayed a 
configurable number of times until it is processed correctly - if it 
fails to be processed correctly after all the redelivery attempts its 
placed on a dead letter queue for processing later on. You don't 
really need to put your own secondary queue system in place. 


-- 

James 
------- 
http://radio.weblogs.com/0112098/ 


------=_Part_6045_11665634.1165588833274--

------=_Part_6044_17819061.1165588833274--