activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Terry K <>
Subject Re: transacted sessions in multi-threaded client
Date Thu, 11 Oct 2007 23:27:08 GMT

Thank you for your comments Joe, 

both senerios you described result in a consumer/session being flooded with
messages up to the prefetch limit. I guess the  transacted multi-threaded
client code would need to evenly iterate the sessions looking for  whatever
sessions have  the messages. Setting the prefetch limit low shoud ensure a
large number of messages are distributed across the sessions while a low
number of messages can reasonabily route to a single session where all
messages get processed in a single thread.

I am pooling my  consumer/sessions and using each one to receive a message
and use it within a thread. Problem is, Apache Commons Pooling does not
allow iteration of items in a pool - to find the session with all the

What is the lowest prefetch limit that would ensure good throughput of
messages without pulling too often from the broker

terry k

ttmdev wrote:
> Here's what I have experienced wrt prefetch limits; you may find it
> useful. 
> If a destination (queue) has active consumers, then as messages arrive at
> the destination, the broker will evenly dispatch the messages across the
> active consumers; no matter what the prefetch limit may be. 
> Now let's say that you have a destination that has a bunch of pending
> messages, but no active consumers to send them to.   If you then activate
> say 5 consumers for that destination, the broker will latch on to the
> first of those 5 that connects to it and will stream the pending messages
> - up to the prefetch limit - to that first consumer. As soon as it
> dis-engages from the first one, it will then begin to 'evenly' dispatch
> messages to all 5 (i.e., assuming the other 4 have already connected w/the
> broker). If the prefetch limit is set to a value that is higher than or
> equal to the number of pending messages, then that first consumer will get
> ALL the messages.    
> Since your message processing time is relatively small, you shouldn't have
> to have a prefetch limit of '1'.   
> Hope this helps,   
> Joe
> Terry K wrote:
>> In a point-to-point queue I want to use transacted sessions to ensure
>> messages are processed completely before being removed from the queue.
>> The client is to be multi threaded to process messages in parallel.
>> I decided each thread would need it's own Session object (for
>> transacting) so that when the work on each message is successfully
>> completed the session can be commited. Session objects would be pooled
>> and retrieved when the next message is ready to be received.
>> The documentation states that, "ActiveMQ is a high performance message
>> bus that streams messages to consumers as fast as possible so that the
>> consumer always has a buffer of messages, in RAM, ready to process -
>> rather than have them explicitly pull messages from the server which adds
>> significant latency per message".
>> The problem is that messages will not be evenly distributed amoungst
>> Sessions, with the likeihood that a single session will be flooded with
>> all the messages making it impossible to distribute the work amoungst
>> multiple threads.
>> The logical next step would be to set the prefetch limit to one and have
>> a pull-one-message-at-a-time dispatch policy  (as described
>> here:
>> Apparently, this is advised against as it's very SLOW
>> ( I expect
>> millions of messages in my queue, each one can be processed in a matter
>> of seconds.
>> The problem of multi threading and transacted sessions must be fairly
>> common - can anyone suggest other options?
>> Terry

View this message in context:
Sent from the ActiveMQ - User mailing list archive at

View raw message