qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gordon Sim <g...@redhat.com>
Subject Re: SubscriptionManager performance problem.
Date Fri, 26 Feb 2010 13:56:39 GMT
On 02/26/2010 10:13 AM, David Stewart wrote:
> The session was synchronous. Using the AsyncSession brought the 75 seconds down to 5-10
which is a fantastic improvement.
> The bottleneck still appears to be the SessionManager though.

SessionManager::subscribe() is synchronous, which will impact the rate. 
One tip is to turn off message flow when you subscribe (use 
SubscriptionSettings(FlowControl::messageWindow(0))) and then add credit 
to resize the window on each subscription after you are set up 
(Subscription::grantMessageCredit() or Session::messageFlow()) to enable 
messages to flow (this can be asynchronous). I get through 20000 in ~25 
secs on my laptop with that (see attached).

However can I ask why you need a separate queue per exchange? Could you 
instead have one queue bound into each of the exchanges? That would be 
far more efficient (both for setup and at runtime) and would still give 
you all the messages sent to any of those exchanges.

>
> I should mention that we're running a vc90 C++ client against a vc90 C++ broker. Could
the broker be the problem?
> Should I see better performance from a linux broker?
>
> -----Original Message-----
> From: Gordon Sim [mailto:gsim@redhat.com]
> Sent: 25 February 2010 17:39
> To: dev@qpid.apache.org
> Cc: David Stewart; users@qpid.apache.org
> Subject: Re: SubscriptionManager performance problem.
>
> On 02/25/2010 04:51 PM, David Stewart wrote:
>> Hi all,
>> we are running a bridge between our old middleware and qpid system which at startup
queries the existing middleware for the number of broadcast groups it knows about. It is a
pricing system so there are ~20000.
>>
>> The bridge creates a fanout exchange for each broadcast group, creates a queue and
binds it to the exchange. All this takes ~75 seconds for 20000. Not a problem.
>>
>> When we add a SubscriptionManager.subscribe() to the loop we get though 1000 requests
in ~3 minutes.
>> I have created an example below which exhibits the problem. The question are we using
the SubscriptionManager incorrectly?
>
> Is oSession an AsyncSession or a SyncSession (a plain Session is a SyncSession)? Each
call to SubscriptionManager::subscribe() will involve a sync() which will certainly reduce
the rate you can get through them, but 1000 in ~3 minutes seems slow even then.
>
>> Is there a better way for us to achieve the result we require?
>>
>> for (int i=0; i<   20000; ++i) {
>>           std::stringstream ss; ss<<   "listener"<<   i;
>>
>>
>>           // Try and declare the exchange. Will succeed even if it already exists.
>>           oSession.exchangeDeclare(qpid::client::arg::exchange=ss.str(),
>>                                           qpid::client::arg::type="fanout",
>>                                           qpid::client::arg::alternateExchange=std::string(),
>>                                           qpid::client::arg::passive=false,
>>
>> qpid::client::arg::durable=true);
>>
>>           oSession.queueDeclare(qpid::client::arg::queue=ss.str(),
>>                                   qpid::client::arg::exclusive=true,
>>                                   qpid::client::arg::autoDelete=false);
>>
>>           oSession.exchangeBind(qpid::client::arg::exchange=ss.str(),
>>                                           qpid::client::arg::queue=ss.str(),
>>
>> qpid::client::arg::bindingKey=ss.str());
>>
>>           oSubscriptionManager.subscribe(*this, ss.str()); }
>>
>> Regards,
>> Dave
>>
>> The information contained in this email is strictly confidential and for the use
of the addressee only, unless otherwise indicated. If you are not the intended recipient,
please do not read, copy, use or disclose to others this message or any attachment. Please
also notify the sender by replying to this email or by telephone (+44 (0)20 7896 0011) and
then delete the email and any copies of it. Opinions, conclusions (etc.) that do not relate
to the official business of this company shall be understood as neither given nor endorsed
by it. IG Index Ltd is a company registered in England and Wales under number 01190902. VAT
registration number 761 2978 07. Registered Office: Friars House, 157-168 Blackfriars Road,
London SE1 8EZ. Authorised and regulated by the Financial Services Authority. FSA Register
number 114059.
>>
>> ---------------------------------------------------------------------
>> Apache Qpid - AMQP Messaging Implementation
>> Project:      http://qpid.apache.org
>> Use/Interact: mailto:dev-subscribe@qpid.apache.org
>>
>
>
> The information contained in this email is strictly confidential and for the use of the
addressee only, unless otherwise indicated. If you are not the intended recipient, please
do not read, copy, use or disclose to others this message or any attachment. Please also notify
the sender by replying to this email or by telephone (+44 (0)20 7896 0011) and then delete
the email and any copies of it. Opinions, conclusions (etc.) that do not relate to the official
business of this company shall be understood as neither given nor endorsed by it. IG Index
Ltd is a company registered in England and Wales under number 01190902. VAT registration number
761 2978 07. Registered Office: Friars House, 157-168 Blackfriars Road, London SE1 8EZ. Authorised
and regulated by the Financial Services Authority. FSA Register number 114059.


Mime
View raw message