activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timothy Bish <>
Subject Re: AMQCPP Openwire much slower than Stomp?
Date Thu, 19 Apr 2007 11:15:07 GMT
On Thu, 2007-04-19 at 09:00 +0200, Albert Strasheim wrote:
> Hello all,
> On Wed, 18 Apr 2007, davidmc wrote:
> <excellent summary snipped>
> > My case was exactly as yours:
> > -the behaviour is different between a local broker and a remote broker,
> > because:
> > *Windows sockets that connect to localhost don't seem to be really sockets,
> > or either, the TCP behaves differently (no Nagle's algorithm).
> > *Linux localhost sockets also behave a bit differently from Linux remote
> > sockets when it comes to Nagle / Delayed ACK issues.
> > -the behaviour is different in Linux and in Windows, because
> > *in windows the Delayed ACK timeout is 200ms. So if your broker is in
> > Windows, you will notice messages with a roundtrip (latency) time of 200ms.
> > *in Linux this timeout is 40ms "only".
> > -message size matters, because big messages fill the buffer and get sent
> > immediately regardless of ACKs received or not.
> > Also I have noticed some strange behaviour with very little packets (less
> > than 512 or so bytes), with those packets, disabling Nagle might not work.
> > The size below which problems happen is different in a Linux local scenario
> > than in a Linux remote scenario.
> This describes almost exactly what I'm seeing! The speed depends 
> greatly on whether I'm using a broker on Linux or Windows and Stomp or 
> Openwire. In order of most to least messages/sec (without TCP_NODELAY 
> being set, as far as I can tell) I get:
> 1. AMQCPP Linux Stomp -> Local Broker Linux
> 2. AMQCPP Linux Stomp -> Remote Broker Windows
> 3. AMQCPP Linux Openwire -> Remote Broker Windows
> 4. AMQCPP Linux Openwire -> Local Broker Linux
> Also, 4 is more than an order of magnitude slower than 1. with 1, 2 and 
> 3 all in the same ballpark. This is with a reasonably large message, 
> that might end up compressing quite well with Openwire.
> From what I can tell, AMQCPP supports a broker URL property for setting 
> TCP_NODELAY, but actually setting it doesn't change anything at socket 
> creation time (although I could be mistaken, I just did a quick browse 
> through the code). Maybe TCP_NODELAY should even be the default?

As of last night I have submitted all the changes to implement allowing
TCP_NODELAY to be set, along with some unit tests.  If you haven't
grabbed trunk in a day or so, you won't have all those changes.

We could default it to on if we find its at the root of some of the
performance issues.

> > How to check if this is really the problem:
> > -Go to AMQCPP source code and add a line to your socket creation code, where
> > you explicitely set TCP_NODELAY to True always (to disable Nagle's
> > algorithm). Rebuild and compare.
> I'll give this a go today.
> > -Capture traffic with Wireshark (former Ethereal), or tcpdump, and see if
> > the broker takes a fixed time (40ms in Linux or 200ms in Windows) to send
> > ACKs back.
> > 
> > Hope this helps :)
> I think it might. Thanks very much.
> Cheers.
> Albert

View raw message