qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Anthony Foglia <AFog...@princeton.com>
Subject Re: Recommended API for Python?
Date Wed, 08 Jun 2011 00:02:02 GMT
On 06/03/2011 02:58 PM, Ted Ross wrote:
> API (3) is a literal/mechanical binding of the C++ API in Python. Our
> intent is to provide a thin Python wrapper around this binding that is a
> drop-in replacement for API(2). The benefits of this API are that all of
> the features of the C++ API are made available via Python (RDMA
> transport, full SASL auth/security support, future features, etc.). I
> would also expect the wrapped API to perform better than the pure-Python
> API. I find it surprising that you're not seeing this.

Well, I've repeated my timings numerous times, and the results seem to 
be that the on average the SWIG library performs slightly faster that 
the Python library.  But the variance is humongous.  I'm only timing the 
consumer side, so maybe the performance will be better on the sender. 
But for my last 5 runs of 20000 messages, I'm getting rates of 1600 +/- 
20 for the pure python api, and 1320 +/- 844 for the swig version.  And 
these are the typical results, comparable times on average, with huge 
variance for the swig api.

One major difference though is that the python version is spending 
nearly all the time in user space, while the swig version is only 
spending a tenth in user, and 5% in system space.  I assume the rest is 
waiting for the network, but (a) I'm trying to acknowledge the messages 
asynchronously, and (b) it would be quite the coincidence if the Python 
client is getting maxed at exactly the same rate as the C++ broker.

Either of these times are way less than the 50,000 transfers/sec 
qpid-perftest tells me the broker is capable of.  (34,000 publish, 
26,000 subscribe).

I've attached the code I'm using in a tarball, in case people have any 
suggestions.


-- 
Anthony Foglia
Princeton Consultants
(609) 987-8787 x233

Mime
View raw message