qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gordon Sim <g...@redhat.com>
Subject Re: C++ broker and large messages - performance issue
Date Fri, 24 Sep 2010 08:26:11 GMT
On 09/23/2010 09:18 PM, Joe Nardone wrote:
>> The process size doesn't grow appreciably when receiving the data
>> frames.  Which makes sense in that it's only dealing with a frame at a
>> time.
>>
>> The trace always hangs up the same way.  It ping/pongs between
>> amqp_0_10::Connection::decode and SessionState::receiverRecord until
>> it gets to frame 79, at which point the delay kicks in.
>>
>
> Ok, I found the limit that is affecting this:
>
> --staging-threshold N (5000000)   Stages messages over N bytes to disk
>
> That explains my magic number.  If I raise the threshold, I can send
> messages less than this size almost immediately.
>
> I also noticed that qpidd, despite me not passing in an explicit
> --load-module for the message store, still picks up my msgstore.so, so
> it is using that as the offload store.  Removing that library makes
> the client work (my apologies earlier, I figured I had to pass it in
> explicitly.)
>
> That all being said -- 3 seconds a frame for the BDB persistence layer
> is underwhelming.  Are there any performance tricks with using that
> module?  Has there been any progress on a AIO implementation for
> flow-to-disk messages?

Yes, flow to disk now uses the AIO journal on linux and BDB is not used 
for persisting messages at all. Also the 'staging' option in its current 
form was removed; it only deals with the publishing side of the problem 
and didn't do a great job of that. You need to have enough memory for 
the largest message you send at present.

---------------------------------------------------------------------
Apache Qpid - AMQP Messaging Implementation
Project:      http://qpid.apache.org
Use/Interact: mailto:users-subscribe@qpid.apache.org


Mime
View raw message