activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Strachan" <>
Subject Re: How to have a highly available and scalable setup using network of brokers?
Date Thu, 21 Jun 2007 06:38:57 GMT
On 6/20/07, Nicky Sandhu <> wrote:
> James.Strachan wrote:
> >
> > On 6/20/07, Nicky Sandhu <> wrote:
> >>
> >> I am in the process of trying to setup a network of brokers for HA
> >
> > If you want HA then you want a Master/Slave arrangement as you
> > probably want to replicate messages to multiple physical brokers (or
> > share state across multiple brokers via JDBC or a SAN)..
> >
> >
> >
> I have that (see the diagram) so I get what you are saying... but
> James.Strachan wrote:
> >
> >
> >> behind a
> >> load balancer. The aim is to allow clients to see a virtual broker (which
> >> is
> >> really load balanced and with fail over) with a single IP/port.
> >
> > FWIW do you really need to load balance across brokers? What kinds of
> > message throughputs are we talking about?
> >
> > i.e. you only need to move from a master/slave to a network of
> > masters/slaves when you have pretty large throughput.
> >
>  ... we need to scale upto 3-4 K msgs/sec with durable subscribes on topics,
> with persistent messages and upwards of a couple hundred publishers and
> couple thousand consumers. Msg sizes range from 1K - 10 M. I am worried
> about memory requirements on a single master broker being able to handle
> that.

It'd be interesting to see what kinds of throughput, RAM & CPU usage
you get in your environment; you might find a single broker can easily
handle that load on a decent 2-4 way blade etc

I guess what I was trying to say was; start out doing the easiest
thing that might work; then only get more complex when you really need

> James.Strachan wrote:
> >
> >> Any alternative ways to achieve
> >> HA/scalability which shields the client from configuration changes as we
> >> scale up?
> >
> > On the client side, its purely about the discovery of brokers -
> > whatever the topology & number of brokers - so you could use one of
> > the existing discovery mechanisms (e.g. zeroconf or multicast) or
> > write your own if you want a non-multicast based one.
> >
> >
> >
>  so let me understand what you are saying. I could potentially write a load
> balancing discovery mechanism  that could connect clients with the least
> stressed out broker at the moment of connection request ? cool !!!

Sure. We've tried to make every aspect of ActiveMQ easily pluggable,
so you can tinker to your hearts content to get the exact messaging
bus of your dreams :). Its pretty simple to plug in any kind of
discovery mechanism you wanna use to find the available brokers (an
LDAP option might be interesting, or a HTTP based one where brokers
POST themselves to a web site & clients GET the list of active brokers
that have heart-beated within a time period etc).

Incidentally, something even cooler - the JEDI transport...

Its not been implemented yet  - I think John Heitmann had a version
somewhere but he never got around to contribute it :(. But its
basically a derivation of the FanOutTransport.

Basically each client connects to all brokers (or all Masters if
you're using Master/Slaves).

Then as commands need to be sent to a broker (sending messages,
performing transactions, acking etc), the client can, at runtime,
choose which broker to use for each *destination*. i.e. the
JediTransport is a kinda CompositeTransport; choosing the actual
transport implementations to send each Command down; for publishing on
a topic, it might be all of them; for publishing on a queue, it'd pick
one broker etc.

So for example; you could then introduce a partiioning mechanism;
where all consumers of a particular destination use a particular
broker. This is kinda like Message Groups, but on the client side
rather than the broker side. A client can basically choose to load
balance its use of destinations across whichever brokers it wishes.
Its also an alternative to store & forward brokers; as the clients can
connect & consume from every broker (so it saves an extra hop).

This is slightly different from the 'pick the broker with the least
traffic for all my messaging load' as you can instead do the load
balancing at a more finely grained level; at each destination rather
than connection.

The main downside of the Jedi transport is the possible increase in
the numbers of connections open on a broker; if you were talking
50,000 connections, then a store/forward network of brokers might be
better; but if its only in the 1-5,000 range, then a single broker can
handle that (providing you configure enough file descriptors on unix).


View raw message