activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Brad Harvey <Brad.Har...@gbst.com>
Subject activemq 5 amqp connector batching does not improve performance
Date Tue, 03 Sep 2019 01:07:10 GMT
Hi,

I have observed that using batches does not improve the performance of ActiveMQ 5's AMQP connector.
 I have found this for consumers and producers using JMS transacted sessions, and also for
consumers with client_acknowledge every N messages.  The performance does not change at all
compared to individual messages.  I suspect that every message is still acked individually.

Is this a known limitation of the amqp connector with activeMQ 5?

This problem does not occur with Active MQ 5 Openwire, nor does it occur with Artemis AMQP.
 After initially observing the problem with Camel sjms-batch + QPID JMS, I was able to use
quiver to reproduce with QPID JMS and QPID messaging CPP.

I have not tested whether the transactional behaviour is correct (i.e. atomic).

Quiver is available here:
https://github.com/ssorj/quiver

I used the quiver docker image.  I was testing with 6600 byte durable messages, "out of the
box" local PC activemq 5.15.9 / artemis 2.9.0 installs.  The problem is also reproduceable
on AmazonMQ.

Following are some example quiver commands and some test results from a laptop - intended
to show relative throughput for no transactions vs transactions rather than any absolute performance
numbers.

Baseline test cases - Openwire & AMQP have similar performance for largish durable messages
when sent individually.

<raw>

# activemq-jms, activemq openwire or artemis openwire, transaction size 1000
quiver --verbose --arrow activemq-jms --body 6600 --count 10000 --durable tcp://hostname:61616/q0

activemq results:

Sender ......................................... activemq-jms
Receiver ....................................... activemq-jms
Address URL ...................... tcp://hostname:61616/q0
Output files ........................... /tmp/quiver-i7gry34k
Count ................................................ 10,000 messages
Body size ............................................. 6,600 bytes
Credit window ......................................... 1,000 messages
Flags ............................................... durable

RESULTS

Count ................................................ 10,000 messages
Duration ............................................... 22.8 seconds
Sender rate ............................................. 438 messages/s
Receiver rate ........................................... 438 messages/s
End-to-end rate ......................................... 438 messages/s


artemis results:

CONFIGURATION

Sender ......................................... activemq-jms
Receiver ....................................... activemq-jms
Address URL ...................... tcp://hostname:61616/q0
Output files ........................... /tmp/quiver-eco6j388
Count ................................................ 10,000 messages
Body size ............................................. 6,600 bytes
Credit window ......................................... 1,000 messages
Flags ............................................... durable

RESULTS

Count ................................................ 10,000 messages
Duration ............................................... 27.1 seconds
Sender rate ............................................. 369 messages/s
Receiver rate ........................................... 369 messages/s
End-to-end rate ......................................... 369 messages/s



# qpid-jms, activemq amqp, transaction size 1000
quiver --verbose --arrow qpid-jms --body 6600 --count 10000 --durable amqp://hostname:5672/q0

CONFIGURATION

Sender ............................................. qpid-jms
Receiver ........................................... qpid-jms
Address URL ...................... amqp://hostname:5672/q0
Output files ........................... /tmp/quiver-fbt12d7g
Count ................................................ 10,000 messages
Body size ............................................. 6,600 bytes
Credit window ......................................... 1,000 messages
Flags ............................................... durable

RESULTS

Count ................................................ 10,000 messages
Duration ............................................... 28.3 seconds
Sender rate ............................................. 353 messages/s
Receiver rate ........................................... 354 messages/s
End-to-end rate ......................................... 353 messages/s



# output of remaining individual tests omitted for brevity, results in same order of magnitude.

# qpid-messaging-cpp, activemq amqp, transaction size 1000
quiver --verbose --arrow qpid-messaging-cpp --body 6600 --count 10000 --durable amqp://hostname:5672/q0

# qpid-jms + artemis amqp, transaction size 1000
quiver --verbose --arrow qpid-jms --body 6600 --count 10000 --durable amqp://hostname:61616/q0

# qpid-messaging-cpp, artemis amqp, transaction size 1000
quiver --verbose --arrow qpid-messaging-cpp --body 6600 --count 10000 --durable amqp://hostname:61616/q0


</raw>

These test cases give expected results - transaction size of 1000 significantly improves performance.


<raw>
# activemq-jms, activemq openwire or artemis openwire, transaction size 1000
quiver --verbose --arrow activemq-jms --body 6600 --transaction-size 1000 --count 10000 --durable
tcp://hostname:61616/q0

activemq:

CONFIGURATION

Sender ......................................... activemq-jms
Receiver ....................................... activemq-jms
Address URL ...................... tcp://hostname:61616/q0
Output files ........................... /tmp/quiver-o893bg1n
Count ................................................ 10,000 messages
Body size ............................................. 6,600 bytes
Credit window ......................................... 1,000 messages
Transaction size ...................................... 1,000 messages
Flags ............................................... durable

RESULTS

Count ................................................ 10,000 messages
Duration ................................................ 3.3 seconds
Sender rate ........................................... 3,393 messages/s
Receiver rate ......................................... 3,204 messages/s
End-to-end rate ....................................... 3,007 messages/s


artemis:

CONFIGURATION

Sender ......................................... activemq-jms
Receiver ....................................... activemq-jms
Address URL ...................... tcp://hostname:61616/q0
Output files ........................... /tmp/quiver-mp8zwfh0
Count ................................................ 10,000 messages
Body size ............................................. 6,600 bytes
Credit window ......................................... 1,000 messages
Transaction size ...................................... 1,000 messages
Flags ............................................... durable

RESULTS

Count ................................................ 10,000 messages
Duration ................................................ 3.7 seconds
Sender rate ........................................... 3,505 messages/s
Receiver rate ......................................... 3,031 messages/s
End-to-end rate ....................................... 2,702 messages/s



# qpid-jms + artemis amqp, transaction size 1000
quiver --verbose --arrow qpid-jms --body 6600 --count 10000 --transaction-size 1000 --durable
amqp://hostname:61616/q0

CONFIGURATION

Sender ............................................. qpid-jms
Receiver ........................................... qpid-jms
Address URL ..................... amqp://hostname:61616/q0
Output files ........................... /tmp/quiver-ok2sjt3s
Count ................................................ 10,000 messages
Body size ............................................. 6,600 bytes
Credit window ......................................... 1,000 messages
Transaction size ...................................... 1,000 messages
Flags ............................................... durable

RESULTS

Count ................................................ 10,000 messages
Duration ................................................ 5.7 seconds
Sender rate ........................................... 2,714 messages/s
Receiver rate ......................................... 2,050 messages/s
End-to-end rate ....................................... 1,767 messages/s


# qpid-messaging-cpp, artemis amqp, transaction size 1000
quiver --verbose --arrow qpid-messaging-cpp --body 6600 --count 10000 --transaction-size 1000
--durable amqp://hostname:61616/q0

CONFIGURATION

Sender ................................... qpid-messaging-cpp
Receiver ................................. qpid-messaging-cpp
Address URL ..................... amqp://hostname:61616/q0
Output files ........................... /tmp/quiver-8gndofrc
Count ................................................ 10,000 messages
Body size ............................................. 6,600 bytes
Credit window ......................................... 1,000 messages
Transaction size ...................................... 1,000 messages
Flags ............................................... durable

RESULTS

Count ................................................ 10,000 messages
Duration ................................................ 3.1 seconds
Sender rate ........................................... 6,039 messages/s
Receiver rate ......................................... 3,351 messages/s
End-to-end rate ....................................... 3,175 messages/s
</raw>

These two test cases give unexpected results - performance roughly equal to the "sent individually"
tests, but expected performance is similar to the other transaction size of 1000 tests.


<raw>
# qpid-jms, activemq amqp, transaction size 1000
quiver --verbose --arrow qpid-jms --body 6600 --transaction-size 1000 --count 10000 --durable
amqp://hostname:5672/q0

CONFIGURATION

Sender ............................................. qpid-jms
Receiver ........................................... qpid-jms
Address URL ...................... amqp://hostname:5672/q0
Output files ........................... /tmp/quiver-0tjnz8sn
Count ................................................ 10,000 messages
Body size ............................................. 6,600 bytes
Credit window ......................................... 1,000 messages
Transaction size ...................................... 1,000 messages
Flags ............................................... durable

RESULTS

Count ................................................ 10,000 messages
Duration ............................................... 26.9 seconds
Sender rate ............................................. 455 messages/s
Receiver rate ........................................... 396 messages/s
End-to-end rate ......................................... 372 messages/s



# qpid-messaging-cpp, activemq amqp, transaction size 1000
quiver --verbose --arrow qpid-messaging-cpp --body 6600 --count 10000 --transaction-size 1000
--durable amqp://hostname:5672/q0

CONFIGURATION

Sender ................................... qpid-messaging-cpp
Receiver ................................. qpid-messaging-cpp
Address URL ...................... amqp://hostname:5672/q0
Output files ........................... /tmp/quiver-y4k_139e
Count ................................................ 10,000 messages
Body size ............................................. 6,600 bytes
Credit window ......................................... 1,000 messages
Transaction size ...................................... 1,000 messages
Flags ............................................... durable

RESULTS

Count ................................................ 10,000 messages
Duration ............................................... 25.9 seconds
Sender rate ............................................. 459 messages/s
Receiver rate ........................................... 410 messages/s
End-to-end rate ......................................... 387 messages/s


</raw>



The information transmitted is intended only for the person or entity to which it is addressed
and may contain confidential and / or privileged material that may be governed by confidential
information provisions contained in the agreement between GBST and your company. Any disclosure,
copying, distribution, or other use without the express consent of the sender is prohibited.
If you received this in error, please contact the sender and delete the material from any
computer. All rights in the information transmitted, including copyright, are reserved. Nothing
in this message should be interpreted as a digital signature that can be used to authenticate
a document. No warranty is given by the sender that any attachments to this email are free
from viruses or other defects.

Mime
View raw message