activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tim Bain <tb...@alumni.duke.edu>
Subject Re: Network Connector Performance
Date Wed, 21 Jan 2015 13:40:33 GMT
As long as no one objects to the idea in the next day or two, you should
submit this enhancement request via JIRA (
https://issues.apache.org/jira/browse/AMQ).
On Jan 21, 2015 1:07 AM, "Ehud Eshet" <ehud.eshet@imperva.com> wrote:

> Hi,
>
> I have a producer that must be able to keep producing even when the remote
> broker is unavailable.
> failover is not enough since the problem might be in the network or even
> the
> local NIC.
>
> Thus, I planed to have a local broker that will use a network connector to
> route all messages to the remote broker.
>
> I use ActiveMQ 5.10 with levelDB persistence on Java SE 1.7 (both local and
> remote servers).
> Average message length is 1KB.
> I need to be able to send at least 10,000 message per second.
>
> Send throughput to the local broker is about 100 messages per second.
> When using transactions and committing every 1000 messages, throughput jump
> to 12,000 messages per second.
>
> The network connector doesn't use transactions.
> Thus, its throughput is only 100 messages per second.
>
> I tried to use proxy connector in order to force useAsyncSend=true on the
> network connector with no success.
>
> Finally, I removed the network connector all together and implemented its
> functionality myself.
> In another thread on the local server I used a transacted session to local
> broker plus a transacted session to the remote broker. Each message
> received
> from the local broker was sent into the remote broker. Every 1000 messages
> (or when more than a second passed since last commit), I committed both
> sessions.
>
> The throughput of the new forwarder thread is 11,000 message per second.
> When running in parallel to the local producer the throughput is still
> 11,000 messages per second.
>
> I would like to suggest two additional properties to networkConnector:
> 1. maxBatchSize - if specified and higher than 1, use transactions to
> forward messages to remote broker.
> 2. maxLatency.
>
> The network connector will send (commit)  up to <maxBatchSize> messages in
> a
> transaction.
> It will commit immediately, if more than <maxLatency> milliseconds passed
> since last commit.
>
>
>
> --
> View this message in context:
> http://activemq.2283324.n4.nabble.com/Network-Connector-Performance-tp4690186.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message