activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ghuillaume <>
Subject Poor performances with large number of consumers
Date Mon, 12 Aug 2019 13:41:48 GMT

I have a network of 2 non-persistent brokers which is dedicated to dispatch
messages to 10000 different consumers using a single topic.
Each consumer use a topic selector to filter messages.
The messages are produced by hundreds of producers : multiple
Java/SpringBoot instances with a CachedConnexionFactory.

I have other queues and topics but it's not the point here.

My problem is that the more I have consumers on my topics, the more I have
latency to deliver messages. I've designed a JMeter test suite to produce
messages and measure performances easily, below the results :
- With no consumer on the topic : up to 10k msg/second can be delivered with
good performances (max/average time in ms reported by Jmeter : 400/0,8).
JMeter is not able to produce more messages, I can't measure performances
with higher throughput.
- With 10K consumers : from 1k msg/second, poor performances (max/average
time in ms reported by Jmeter : 30000/31). I have good performances with
lower throughput

All my clients (producers and consumers) are connected on the 2 brokers
using failover transport reference, so I don't know on which broker the
messages will be produced to and I don't know on which broker a message need
to be forwarded to be consumed.

You will find my config in attachment, and below some details about it :
- conduitSubscriptions=false on networkConnector : because I use topic
- advisorySupport=false : because if true it doubles the number of consumers
on my topic in a 2 nodes network scenario (i.e. : if I have 10K consumers, I
will have 10K on each instance instead of 5K per instance), which impact
negatively the performances
- All my topics and queues are "staticallyIncludedDestinations" : because
dynamic require advisorySupport
- MaxPageSize=20000 on the topic policyEntry : it seems to be efficient. I
think I need a large number to be able to evaluate a large amout of topic
selector at the same time
- constantPendingMessageLimitStrategy=20 on the topic because I have a lot
of consumers (20 x 10000 = 2000000 pending messages)
- The two brokers declare the same transport connector so it results in a
duplex bridge between them.

I'm using ActiveMQ 5.15.3


Sent from:

View raw message