qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Alan Conway <acon...@redhat.com>
Subject [dispatch] router concurrency and scale [was Re: [Performance] Benchmarking Qpid dispatch router 0.6.0 with Qpid Java Broker 6.0.0]
Date Mon, 08 Aug 2016 16:51:27 GMT
On Tue, 2016-08-02 at 14:44 -0400, Ted Ross wrote:
> 
> On 08/02/2016 02:10 PM, Adel Boutros wrote:
[snip]
> > 
> What you both explained to me about the single connection is indeed
> > a plausible candidate because in the tests of "broker only", the
> > throughput of a single connection is around 3 500 msg/s. So on a
> > single connection, I shouldn't go above that figure which is what I
> > am seeing. I imagine that when I add more producers/consumers, the
> > throughput will shrink even more because the same connection is
> > used by all the producers and the consumers.
> > 
> > Do you think it might be an a good idea if the connections were per
> > workerThread and not only a single connection?
> 
> I think this is an interesting feature to consider

I think we should consider this. For dispatch the big question is not what rate the back-end
can handle, but "how does adding dispatch affect performance and scalability?" Say the back-end
only does 5k msg/s on a single connection, but can do 15k on 3 connections. If dispatch reduces
that to 5k *no matter how many client connections* that is a problem. 

Perhaps connectors should have a "concurrency" setting. I wouldn't tie it to the router's
worker threads because it is really about concurrency at the back-end, not on the router.
We need to avoid unexpected re-ordering of messages between a given client/back-end pair,
so we have to think about how to bind/balance client load over back-end connections. It complicates
the disconnect/reconnect/am-I-connected story quite a bit.

Not simple, but definitely worth a think...

Cheers,
Alan.





---------------------------------------------------------------------
To unsubscribe, e-mail: users-unsubscribe@qpid.apache.org
For additional commands, e-mail: users-help@qpid.apache.org


Mime
View raw message