activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dejan Bosanac <de...@nighttale.net>
Subject Re: Configuring Distributed Queues in Store/Forward Network
Date Mon, 11 Oct 2010 08:16:02 GMT
You can set prefetchSize property on the network connector, which
should help with your use case.
See http://activemq.apache.org/networks-of-brokers.html for more details.

Cheers
--
Dejan Bosanac - http://twitter.com/dejanb

Open Source Integration - http://fusesource.com/
ActiveMQ in Action - http://www.manning.com/snyder/
Blog - http://www.nighttale.net



On Fri, Oct 8, 2010 at 8:04 PM, schow <schow@alarmpoint.com> wrote:
>
>> The prefetch limit determines how many messages can be sent to a
>> consumer to be processed. The default value for the queue prefetch
>> limit is 1000. Oftentimes this limit is fine for fast consumers that
>> never slow down, but it can easily flood a slow consumer. This is why
>> we recommend adjusting the prefetch limit based on the consumer
>> behavior.
>
> Hi Bruce,
>
> We've noticed the same behaviour as the OP and found that setting a prefetch
> limit of 1 on the network connector URL had no effect in limiting/balancing
> the message distribution.  The reason was because the
> DemandForwardingBridgeSupport creates a "proxy" consumer for its local
> instance of the shared queue.  This proxy consumer is what gets registered
> with the producer's instance of the shared queue and its only job is to
> receive and store the messages.  Since the proxy consumer is detached from
> the actual consumer, with a prefetch of 1 it just spins as fast as possible
> receiving and storing the messages --- the only difference is that it sends
> back on ACK on each message as opposed to every 1000 messages.
>
> Is this your understanding as well regarding the effect of prefetch in a
> network of brokers?  I'm guessing it's the reason why you suggested a custom
> dispatch policy?
>
> A similarly behaving scenario is when a producer generates say 10,000
> messages into a shared queue and there are no consumers.  Two consumers are
> then started at roughly the same time.  We'd hoped that the consumers would
> a roughly equal number of messages, but instead we noticed that one consumer
> would tend to get all the messages.  Again, the reason was that one consumer
> manages to start up slightly ahead of the other and the
> DemandForwardingBridgeSupport "proxy" consumer gobbles up all available
> messages as quickly as it can, regardless of the actual spead of its
> "actual" consumer.
>
> Unfortunately, this behaviour has been very problematic for us in regards to
> using distributed queues.  We're hoping that there will be enough interest
> from other AMQ users to warrant the work to enhance the dispatching
> mechanism.
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/Configuring-Distributed-Queues-in-Store-Forward-Network-tp2967501p2968819.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

Mime
View raw message