activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Davies <>
Subject Re: AMQ-1866: Muliple consumers on a queue and only run as fast as the slowest consumer
Date Fri, 25 Jul 2008 18:11:45 GMT
Yep - that's the behaviour of Queues before - however it resulted in  
very poor performance for high throughput systems, as there was a lot  
of contention around locks, reference counts and garbage collection -  
let me think about it abit more


On 25 Jul 2008, at 17:31, Hiram Chirino wrote:

> Well,
> I've created a fix for the problem and attached it to the issue as a  
> patch:
> If you guys get a chance please review it before I commit it.  I want
> to make sure I'm not overseeing something else.  So the patch changes
> the section in the dispatching logic where it has determined that all
> consumer's prefetches were full.  It used to try to figure out which
> consumer was the less loaded consumer and enqueue the message only to
> his subscription.  Unfortunately it was sometimes figuring out that it
> should dispatch to the slow consumer.  Once that happened, that page
> table message would get locked up and not get consumed for a while.
> The patch now enqueues the message to all subscriptions and depends on
> the fact that the subscriptions 'lock' the message before dispatching
> to the clients.  The next effect is the message will eventually get
> dispatched and consumed by the fast consumer since the slow consumer
> does not have an exclusive lock on that message.
> The performance implication I think is that enqueuing the message to
> all consumers is more work than just picking one to enqueue to so, if
> you have lots of consumes and they all tend to have their prefetches
> full, then the performance will suffer a little bit more than the
> current implementation.  There might be some memory implications too
> since, the slow consumer is going to have enqueued messages that have
> been locked and dispatched by faster consumers.  And he only clears
> those messages out of his list when he iterates it looking for new
> messages to dispatch which only occurs when he acks.
> Anyways looking for feedback on the patch, if we can do this in a
> better way please let me know.
> Regards,
> Hiram
> On Fri, Jul 25, 2008 at 11:28 AM, Hiram Chirino < 
> > wrote:
>> Hi Guys,
>> I looking for feed back and potential solutions for AMQ-1866:
>> I think the issue revolves around how ActiveMQ 5.x pages and
>> dispatches messages to consumers. Every queue keeps list of paged in
>> messages which defaults to 100 max entires. It round robins
>> dispatching messages between consumers, even the slow consumers. Once
>> the slow consumer gets 100 dispatched messages which it has not yet
>> acked due to it being slow, then no further messages are paged in
>> since the page in list is full.
>> It would be nice if someone else can confirm my analysis of the
>> problem before we start diving into possible ways of fixing it.
>> --
>> Regards,
>> Hiram
>> Blog:
>> Open Source SOA
> -- 
> Regards,
> Hiram
> Blog:
> Open Source SOA

View raw message