activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Rob Lugt <robert.l...@cheynecapital.com>
Subject Re: Topic consumer appears throttled
Date Sat, 23 Sep 2006 22:29:51 GMT

I modified the test program to call IMessage.Acknowledge, but unfortunately
it didn't make any difference.

One thing I've discovered that does make a difference is if the broker is
moved onto the local machine. When all three components (broker, publisher
and consumer) are on one machine, the message rate is fast and constant.
Unfortunately this is no real help because it doesn't match our target
environment. 

I'm going to take a wild stab in the dark...  Is it possible that my
consumer is being identified as a "slow consumer", and therefore being given
messages more slowly?  I'm completely ignorant to the internal workings of
the AMQ broker, so have little idea if this is likely - but I did read the
article [1] which indicates slow consumers are given special attention and
so the symptoms could possibly be explained by it.  If this does turn out to
be the case, it remains strange for (at least) two reasons: 
1) the consumer is actually pretty fast, but even though it does nothing
with the received message it can't consume quite consume at the rate that
the producer can produce. 
2) it seems the consumer slows down as soon as there are >1000 messages
pending.  I would expect this slowdown to occur at 32,766 which is, I
understand, the prefetch limit for a non-durable topic[2].

Anyway, to test this theory I added "?consumer.prefetchSize=2000" to the
TOPIC name in the consumer.  This made no difference :-(  Can you confirm
that all destination options are transported to the broker via NMS/OpenWire?

Another related thought struck me. How can I be sure that I've created a
non-durable Topic?  I use the ISession.GetTopic() method which magically
creates the topic for me. I presume the result is a non-durable topic?

Thanks for the suggestions.  Hopefully I've given a clearer picture of the
problem.

Best regards
~Rob Lugt

[1] http://incubator.apache.org/activemq/slow-consumer-handling.html
[2] http://www.activemq.org/site/performance-tuning.html


James.Strachan wrote:
> 
> I wonder have you tried calling IMessage.Acknowledge()?
> 
> On 9/22/06, Rob Lugt <robert.lugt@cheynecapital.com> wrote:
>>
>> Quick update to my earlier message.  I did eventually find the overload
>> on
>> IConnection.CreateSession() to pass an AcknowledgementMode.  I tried
>> setting
>> this to either AutoAcknowledge or DupsOkAcknowledge, but neither had an
>> affect on the apparent throttling of the consumer.
>>
>> ~Rob
>>
>>
>> Rob Lugt wrote:
>> >
>> > Hi James, thanks for the quick reply.
>> >
>> > Yes, I'm using NMS and C# for the client.  I wasn't specifying any
>> > particular acknowledgement mode, so my previous tests would have been
>> > using the default - but your question prompted me to try a few things
>> out.
>> >
>> > I found this article which describes acknowledgement modes for
>> consumers:
>> >
>> http://www.activemq.org/site/activemq-performance-module-users-manual.html
>> >
>> > I couldn't find a way to directly manipulate the consumer
>> acknowledgement
>> > mode using the NMS classes (if there is a way please tell me), so I
>> used
>> > the URI syntax on the GetTopic() method e.g.
>> >
>> >   IDestination destination =
>> >
>> session.GetTopic("PROD.PRICES.LIVE.IBM_LN?consumer.sessAckMode=autoAck");
>> >
>> > Sorry, I don't know if I've got the syntax right, but it didn't make
>> any
>> > difference to the tests.  I also tried the value "dupsAck" because I
>> read
>> > somewhere that this could improve performance but it also didn't have
>> any
>> > affect.
>> >
>> > The listener is async (delegate added to consumer.Listener) - and is
>> very
>> > light-weight.  I checked the CPU and memory activity on the client and
>> the
>> > server and both were comfortably under utilised.
>> >
>> > Best regards
>> > Rob Lugt
>> >
>> >
>> > James.Strachan wrote:
>> >>
>> >> This sounds like it could be related to acknowledgements - you're
>> >> using NMS and C# as the client right? What acknowledgement mode are
>> >> you using and are you using the async or sync listener?
>> >>
>> >> On 9/22/06, Rob Lugt <robert.lugt@cheynecapital.com> wrote:
>> >>>
>> >>> Hi
>> >>>
>> >>> I'm hoping to use AMQ for a stock price publishing system, so high
>> >>> throughput and low latency are important factors.  To verify that the
>> >>> kind
>> >>> of throughput we are after is achievable I've created a very simple
>> >>> producer/consumer pair based on the sample code in
>> >>> http://www.activemq.org/site/nms.html.
>> >>>
>> >>> I've come across a potential problem where the consumer's receive
>> rate
>> >>> dramatically falls after the number of enqueued messages exceeds a
>> >>> certain
>> >>> threshold.  I'm pretty sure I must be falling foul of a configuration
>> >>> problem because the number 1000 seems to be significant.
>> >>>
>> >>> If the publisher sends 1000 messages, the consumer happilly receives
>> >>> them in
>> >>> a couple fo seconds. But, if I configure the publisher to send 1500
>> >>> messages
>> >>> in quick succession, I see a perculiar pattern: the consumer receives
>> >>> the
>> >>> first 300 messages quickly, it then receives the next 200 messages
>> very
>> >>> slowly (approx 5 msgs/s) until there are exactly 1000 messages left
>> in
>> >>> the
>> >>> queue and then it speeds through the last 1000 in just 2 seconds. If
>> I
>> >>> increase the number of messages sent, the same pattern remains i.e.
>> the
>> >>> consumer runs slowly until only 1000 messages remain then speeds up!
>> >>>
>> >>> Topology:
>> >>>   Client 1 and Client 2 both running on a single workstation,
>> connecting
>> >>> via
>> >>> OpenWire using NMS
>> >>>   AMQ 4.0.1 Broker running on separate machine (Windows 2003 Server,
>> >>> J2SE
>> >>> 1.5.0_06)
>> >>>
>> >>> Client 1 opens a session and publishes n (non-persistent) messages to
>> a
>> >>> single topic.
>> >>> Client 2 opens a session and registers a listener on the same topic.
>> >>>
>> >>> I've tried various options to overcome this, none of which seem to
>> have
>> >>> had
>> >>> any impact.
>> >>> On connection uri I have:
>> >>> jms.useAsyncSend=true&consumer.dispatchAsync=false
>> >>> On Topic name I have:
>> jms.useAsyncSend=true&consumer.dispatchAsync=false
>> >>>
>> >>> Any ideas/solutions gratefully received.
>> >>>
>> >>> Best regards
>> >>> Rob Lugt
>> >>>
>> >>>
>> >>>
>> >>> --
>> >>> View this message in context:
>> >>>
>> http://www.nabble.com/Topic-consumer-appears-throttled-tf2318926.html#a6451007
>> >>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>> >>>
>> >>>
>> >>
>> >>
>> >> --
>> >>
>> >> James
>> >> -------
>> >> http://radio.weblogs.com/0112098/
>> >>
>> >>
>> >
>> >
>>
>> --
>> View this message in context:
>> http://www.nabble.com/Topic-consumer-appears-throttled-tf2318926.html#a6455875
>> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>>
>>
> 
> 
> -- 
> 
> James
> -------
> http://radio.weblogs.com/0112098/
> 
> 

-- 
View this message in context: http://www.nabble.com/Topic-consumer-appears-throttled-tf2318926.html#a6467217
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Mime
View raw message