activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ehud Eshet <ehud.es...@imperva.com>
Subject Network Connector Performance
Date Wed, 21 Jan 2015 08:03:30 GMT
Hi,

I have a producer that must be able to keep producing even when the remote
broker is unavailable.
failover is not enough since the problem might be in the network or even the
local NIC.

Thus, I planed to have a local broker that will use a network connector to
route all messages to the remote broker.

I use ActiveMQ 5.10 with levelDB persistence on Java SE 1.7 (both local and
remote servers).
Average message length is 1KB.
I need to be able to send at least 10,000 message per second.

Send throughput to the local broker is about 100 messages per second.
When using transactions and committing every 1000 messages, throughput jump
to 12,000 messages per second.

The network connector doesn't use transactions.
Thus, its throughput is only 100 messages per second.

I tried to use proxy connector in order to force useAsyncSend=true on the
network connector with no success.

Finally, I removed the network connector all together and implemented its
functionality myself.
In another thread on the local server I used a transacted session to local
broker plus a transacted session to the remote broker. Each message received
from the local broker was sent into the remote broker. Every 1000 messages
(or when more than a second passed since last commit), I committed both
sessions.

The throughput of the new forwarder thread is 11,000 message per second.
When running in parallel to the local producer the throughput is still
11,000 messages per second.

I would like to suggest two additional properties to networkConnector:
1. maxBatchSize - if specified and higher than 1, use transactions to
forward messages to remote broker.
2. maxLatency.

The network connector will send (commit)  up to <maxBatchSize> messages in a
transaction.
It will commit immediately, if more than <maxLatency> milliseconds passed
since last commit. 



--
View this message in context: http://activemq.2283324.n4.nabble.com/Network-Connector-Performance-tp4690186.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Mime
View raw message