activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Fernandez <joe.fernan...@ttmsolutions.com>
Subject Re: Scalability of 'Networks of Brokers'
Date Fri, 05 Sep 2008 18:31:29 GMT

Instead of using such a potentially large network of 'embedded' brokers,
could you use a smaller cluster of 'standalone' brokers? A single broker is
capable of handling many clients and throwing in a master/slave
configuration along with 'failover' connectors for the clients should
provide the necessary high availability. 

Joe
http://www.ttmsolutions.com   


Martin_Cornelius wrote:
> 
> Hi all,
> <p>
> i am currently evaluating the scalability of Networks of ActiveMQ Brokers
> with a large number of brokers. The actual drive behind this evaluation: I
> have to design a distributed system that is completely serverless. It
> shall be possible to switch of any of the machines in the system without
> alloying the distributed functionality. The number of active machines in
> the system might vary from only 2 to about 100.<p>
> My current idea: Have an (presumably embedded) ActiveMQ Broker running on
> any of the machines, and let all this Brokers form a 'Network of Brokers'
> (abbreviated NOB in the follwing)
> <p>
> So far, i found that in an NOB a message is forwarded only to those
> brokers where at least one client has subscribed to the topic resp. queue
> the message is sent to. Really fine. If a client subscribes to a queue,
> the broker where the subscription is made sends a single short
> control-message to all other brokers of the NOB. That would still allow
> for thousands of brokers, fine again.
> <p>
> However, i just realized one fact that may compromise scalabilty: If a
> client subscribes to a  topic, the broker where the subscription is done
> sends a not-so-short control-message to all other brokers. So far not too
> bad, but now comes the problem: Every broker seems to react on this
> control-message by sending another not-so-short control-message to all
> other brokers. This means, if N brokers are present, every subscription to
> a topic results in about (N-1)*N*2 control-messages (The additional factor
> two comes from the fact, that the final recipient-broker seems to send
> another response control-message). BTW, The length of the control-messages
> being sent seems to be closely related to the string length of the topic
> name.
> <p>
> I have verified this behaviour with only up to 6 brokers, so i'm not
> totally shure if it will extrapolate to a larger number, but i suspect it
> does. This would mean, that with e.g. 100 Brokers every subscription to a
> topic would result in 100*99*2 control messages being sent. If a single
> control message has about 400 bytes (what i observed), this means a single
> subscription to a topic would result in about 8 Megabyte network traffic.
> This might even be sustainable, but the quadratic order kills scalabilty
> -- with 1000 Brokers we would have to transfer about 1 Gigabyte, what is
> no longer feasible.
> <p>
> Finally, my questions: Is there any configuration option by which this
> quadratic order can be avoided, is my idea simply foolish, or are my
> conclusions wrong ? Note that i gained my conclusions more or less
> 'empirically' using wireshark.
> <p>
> thanks for any advice, Martin
> 
> 

-- 
View this message in context: http://www.nabble.com/Scalability-of-%27Networks-of-Brokers%27-tp19335699p19337053.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Mime
View raw message