activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hiram Chirino" <hi...@hiramchirino.com>
Subject Re: AMQ-1866: Muliple consumers on a queue and only run as fast as the slowest consumer
Date Fri, 25 Jul 2008 16:31:10 GMT
Well,

I've created a fix for the problem and attached it to the issue as a patch:
https://issues.apache.org/activemq/secure/attachment/16781/AMQ1866.patch

If you guys get a chance please review it before I commit it.  I want
to make sure I'm not overseeing something else.  So the patch changes
the section in the dispatching logic where it has determined that all
consumer's prefetches were full.  It used to try to figure out which
consumer was the less loaded consumer and enqueue the message only to
his subscription.  Unfortunately it was sometimes figuring out that it
should dispatch to the slow consumer.  Once that happened, that page
table message would get locked up and not get consumed for a while.
The patch now enqueues the message to all subscriptions and depends on
the fact that the subscriptions 'lock' the message before dispatching
to the clients.  The next effect is the message will eventually get
dispatched and consumed by the fast consumer since the slow consumer
does not have an exclusive lock on that message.

The performance implication I think is that enqueuing the message to
all consumers is more work than just picking one to enqueue to so, if
you have lots of consumes and they all tend to have their prefetches
full, then the performance will suffer a little bit more than the
current implementation.  There might be some memory implications too
since, the slow consumer is going to have enqueued messages that have
been locked and dispatched by faster consumers.  And he only clears
those messages out of his list when he iterates it looking for new
messages to dispatch which only occurs when he acks.

Anyways looking for feedback on the patch, if we can do this in a
better way please let me know.

Regards,
Hiram

On Fri, Jul 25, 2008 at 11:28 AM, Hiram Chirino <hiram@hiramchirino.com> wrote:
> Hi Guys,
>
> I looking for feed back and potential solutions for AMQ-1866:
>
> I think the issue revolves around how ActiveMQ 5.x pages and
> dispatches messages to consumers. Every queue keeps list of paged in
> messages which defaults to 100 max entires. It round robins
> dispatching messages between consumers, even the slow consumers. Once
> the slow consumer gets 100 dispatched messages which it has not yet
> acked due to it being slow, then no further messages are paged in
> since the page in list is full.
>
> It would be nice if someone else can confirm my analysis of the
> problem before we start diving into possible ways of fixing it.
>
> --
> Regards,
> Hiram
>
> Blog: http://hiramchirino.com
>
> Open Source SOA
> http://open.iona.com
>



-- 
Regards,
Hiram

Blog: http://hiramchirino.com

Open Source SOA
http://open.iona.com

Mime
View raw message