Mailing-List: contact activemq-dev-help@geronimo.apache.org; run by ezmlm
Precedence: bulk
Reply-To: activemq-dev@geronimo.apache.org
Received-SPF: pass (asf.osuosl.org: domain of james.strachan@gmail.com
 designates 72.14.214.206 as permitted sender)
DomainKey-Signature: a=rsa-sha1; q=dns; c=nofws;
        s=beta; d=gmail.com;
        h=received:message-id:date:from:to:subject:mime-version:content-type:content-transfer-encoding:content-disposition;
        b=s58F+e1z9IOhyos9dkeDGKff4Fl3F4LGnK8YfU/SARMypf7axn77IHykycWbVmLOQI8gXyQBjwt2ck12e6tT6WSW9B7ZXlo3glOvbaAXiOuECs7eyT+YsH4ZZcbypCDx52PQj1C5oVUEo4xCCwC27o0Rs2e9SABvb/Laah9t+3o=
Message-ID: <ec6e67fd0607031150l65f94fbcj4b46b35083d318e1@mail.gmail.com>
Date: Mon, 3 Jul 2006 19:50:01 +0100
From: "James Strachan" <james.strachan@gmail.com>
To: activemq-dev@geronimo.apache.org
Subject: handling slow consumers on non-persistent topics and AMQ-688
MIME-Version: 1.0
Content-Type: text/plain; charset=ISO-8859-1; format=flowed
Content-Transfer-Encoding: 7bit
Content-Disposition: inline

While perusing JIRA I spotted this issue again...
http://issues.apache.org/activemq/browse/AMQ-688
I know its an issue close to folks at Amazon's hearts.

Dealing with slow consumers is a fascinating problem for a messaging
system; its quite a tricky problem :). Here's some background on the
issue...
http://incubator.apache.org/activemq/slow-consumers.html

together with the currently supported features - to allow messages to
be discarded on slow consumers using a pluggable algorithm...
http://incubator.apache.org/activemq/slow-consumer-handling.html

Now for all consumers we fill up prefetch buffers as quickly as possible...
http://incubator.apache.org/activemq/what-is-the-prefetch-limit-for.html

so there's always a buffer of messages per consumer. For non-durable
topics once these messages are written to a socket they are evicted
from RAM; so we already have some support for slow consumers.

I wanted to start a discussion on both AMQ-688 and to see if folks had
other requirements for handling slow consumers & to try decide what
features & stragegies we should add next in this area.

One of the first requirements folks ask for is that rather than
blocking permanently the non-persistent topic engine until RAM is
cleared, that at a certain threshhold we start spooling to disk. I've
raised a separate JIRA issue for this specific feature request...

http://issues.apache.org/activemq/browse/AMQ-791

Another issue some folks have hit in the past is that for high
performance and to minimise context switches in the broker, we tend to
use the current thread in the broker to dispatch to all the
non-durable topic consumers so a slow/blocked consumer can appear to
'block' the producer.

I've raised this issue to track that feature
http://issues.apache.org/activemq/browse/AMQ-792

I just wondered if folks had any other issues or requirements with the
whole slow consumers and non-durable topics they'd like to discuss? Is
there any requirements we won't have covered if the above two JIRAs
are fixed
-- 

James
-------
http://radio.weblogs.com/0112098/