activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bruce Snyder <bruce.sny...@gmail.com>
Subject Re: Large amounts of queues and topics
Date Mon, 25 Apr 2011 17:11:38 GMT
On Sun, Apr 24, 2011 at 6:58 AM, ripienaar <rip@devco.net> wrote:
> I am working on a system that has large amounts of topics often connected in
> a network of brokers.  Clients are mostly using the Stomp protocol.  I am
> now looking at also adding potentially a lot of queues and was seeking some
> design advice on scalability.
>
> For me the most important characteristic is that message latency should be
> low even for 1000s of subscribes, this works well today with just topics and
> subscribers subscribing to a topic of messages they want.
>
> I am concerned that adding a lot - 1000s - of queues would impact
> performance a lot due to the typical 1 thread per queue, topic and
> subscriber (I know these are tunable and I can use NIO).
>
> I can now redesign things so that instead of many many topics and queues I
> use headers and JMS selectors when subscribing and publishing, this way I
> can have only a hand full of queues and topics with subscribers maintaining
> many subscriptions on these queues/topics.
>
> It's not clear which of these two strategies will work best in the long run
> and so was wondering what would be the most resource intensive on the broker
> and the network of brokers wrt to threads, cpu, memory etc. and what would
> be the best practice for this kind of application keeping in mind message
> latency is very important.

I'm assuming that you have read the following page on the ActiveMQ website:

http://activemq.apache.org/how-do-i-configure-10s-of-1000s-of-queues-in-a-single-broker-.html

Instead of thousands of destinations or thousands of JMS selectors, my
experience has taught me that a mix of the two is the best approach
(especially in a situation where you have thousands of JMS clients
connecting to the network of brokers). Although you can tune ActiveMQ
to handle it, thousands of destinations tend to tax the broker
unnecessarily, especially if you can send messages on the same
destination and partition those messages using JMS selectors and/or
message groups. But remember that selectors must be applied to every
message that flows through the broker, so there is a tipping point for
them as well. I have achieved the best results by dividing messages
across a smaller number of destinations using a smaller number of
selectors. Finding the appropriate balance requires experimentation in
your environment with lifelike load testing.

Bruce
-- 
perl -e 'print unpack("u30","D0G)U8V4\@4VYY9&5R\"F)R=6-E+G-N>61E<D\!G;6%I;\"YC;VT*"
);'

ActiveMQ in Action: http://bit.ly/2je6cQ
Blog: http://bruceblog.org/
Twitter: http://twitter.com/brucesnyder

Mime
View raw message