activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From schow <sc...@alarmpoint.com>
Subject Re: Configuring Distributed Queues in Store/Forward Network
Date Fri, 08 Oct 2010 18:04:33 GMT

> The prefetch limit determines how many messages can be sent to a
> consumer to be processed. The default value for the queue prefetch
> limit is 1000. Oftentimes this limit is fine for fast consumers that
> never slow down, but it can easily flood a slow consumer. This is why
> we recommend adjusting the prefetch limit based on the consumer
> behavior.

Hi Bruce,

We've noticed the same behaviour as the OP and found that setting a prefetch
limit of 1 on the network connector URL had no effect in limiting/balancing
the message distribution.  The reason was because the
DemandForwardingBridgeSupport creates a "proxy" consumer for its local
instance of the shared queue.  This proxy consumer is what gets registered
with the producer's instance of the shared queue and its only job is to
receive and store the messages.  Since the proxy consumer is detached from
the actual consumer, with a prefetch of 1 it just spins as fast as possible
receiving and storing the messages --- the only difference is that it sends
back on ACK on each message as opposed to every 1000 messages.  

Is this your understanding as well regarding the effect of prefetch in a
network of brokers?  I'm guessing it's the reason why you suggested a custom
dispatch policy?

A similarly behaving scenario is when a producer generates say 10,000
messages into a shared queue and there are no consumers.  Two consumers are
then started at roughly the same time.  We'd hoped that the consumers would
a roughly equal number of messages, but instead we noticed that one consumer
would tend to get all the messages.  Again, the reason was that one consumer
manages to start up slightly ahead of the other and the
DemandForwardingBridgeSupport "proxy" consumer gobbles up all available
messages as quickly as it can, regardless of the actual spead of its
"actual" consumer.

Unfortunately, this behaviour has been very problematic for us in regards to
using distributed queues.  We're hoping that there will be enough interest
from other AMQ users to warrant the work to enhance the dispatching
mechanism.
-- 
View this message in context: http://activemq.2283324.n4.nabble.com/Configuring-Distributed-Queues-in-Store-Forward-Network-tp2967501p2968819.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Mime
View raw message