activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ttmdev <joe.fernan...@ttmsolutions.com>
Subject Re: transacted sessions in multi-threaded client
Date Thu, 11 Oct 2007 17:00:57 GMT

Here's what I have experienced wrt prefetch limits; you may find it useful. 

If a destination (queue) has active consumers, then as messages arrive at
the destination, the broker will evenly dispatch the messages across the
active consumers; no matter what the prefetch limit may be. 

Now let's say that you have a destination that has a bunch of pending
messages, but no active consumers to send them to.   If you then activate
say 5 consumers for that destination, the broker will latch on to the first
of those 5 that connects to it and will stream the pending messages - up to
the prefetch limit - to that first consumer. As soon as it dis-engages from
the first one, it will then begin to 'evenly' dispatch messages to all 5
(i.e., assuming the other 4 have already connected w/the broker). If the
prefetch limit is set to a value that is higher than or equal to the number
of pending messages, then that first consumer will get ALL the messages.    

Since your message processing time is relatively small, you shouldn't have
to have a prefetch limit of '1'.   

Hope this helps,   
Joe


 

Terry K wrote:
> 
> In a point-to-point queue I want to use transacted sessions to ensure
> messages are processed completely before being removed from the queue. The
> client is to be multi threaded to process messages in parallel.
> 
> I decided each thread would need it's own Session object (for transacting)
> so that when the work on each message is successfully completed the
> session can be commited. Session objects would be pooled and retrieved
> when the next message is ready to be received.
> 
> The documentation states that, "ActiveMQ is a high performance message bus
> that streams messages to consumers as fast as possible so that the
> consumer always has a buffer of messages, in RAM, ready to process -
> rather than have them explicitly pull messages from the server which adds
> significant latency per message".
> 
> The problem is that messages will not be evenly distributed amoungst
> Sessions, with the likeihood that a single session will be flooded with
> all the messages making it impossible to distribute the work amoungst
> multiple threads.
> 
> The logical next step would be to set the prefetch limit to one and have a
> pull-one-message-at-a-time dispatch policy  (as described
> here:http://activemq.apache.org/what-is-the-prefetch-limit-for.html).
> Apparently, this is advised against as it's very SLOW
> (http://osdir.com/ml/java.activemq.user/2005-03/msg00188.html). I expect
> millions of messages in my queue, each one can be processed in a matter of
> seconds.
> 
> The problem of multi threading and transacted sessions must be fairly
> common - can anyone suggest other options?
> 
> Terry
> 

-- 
View this message in context: http://www.nabble.com/transacted-sessions-in-multi-threaded-client-tf4603908s2354.html#a13160225
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Mime
View raw message