activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "James Strachan" <james.strac...@gmail.com>
Subject Re: PooledConnectionFactory - Spring - Loadbalancing in a network
Date Fri, 07 Sep 2007 03:00:55 GMT
On 9/6/07, Olivier OTTAVI <olivier.ottavi@gmail.com> wrote:
> That will result in easier connection managment between the nodes of the
> network and even load of each brokers. I would like to be able to scale the
> network of brokers without scaling automatically the producers. Like 2
> producers and 3 brokers etc. Is Iona recommand architecture where load
> balancing is done by assigning producers with specific broker ? Beside
> thanks for the information on the average load that a broker can assume.
>

The worry of using a store and forward network of brokers as a way to
load balance load (when usually the broker isn't the node with the
load issue), is that if you're not careful the load actually increases
on the brokers as brokers have to forward messages to other brokers;
so reducing the actual load they can handle to real consumers.
(ActiveMQ does give priority to local consumers to try avoid this
though).

FWIW a simpler approach to do what you want -to load messaging traffic
dynamically across brokers - has been described on and off as the
"jedi" transport...

http://www.nabble.com/Load-Balancing-with-Single-Consumer-tf2252613s2354.html#a6260227
http://issues.apache.org/activemq/browse/AMQ-816

Its basically an extension of the FanOut transport where each JMS
client connects to multiple brokers and subscribing directly to each
broker and sending messages to either one broker chosen possibly
randomly for queue operations, or to all brokers for topic style
operations.

This will scale much better as it avoids the problems of store/forward
(namely having to forward messages from broker to broker); the only
real cost of this approach is the increase in the number of
connections to each broker. But these days, if you enable enough file
descriptors; brokers can handle many thousands of concurrent
connections with ease. (If you've an insanely big network and load you
might wanna use store and forward between JEDI domains).

If you fancy taking a stab at implementing the JEDI transport, by
deriving (or just copy/pasting) from the FanOut transport am sure it
wouldn't be too hard to do & folks on the dev list would definitely
help you out - many of us have wanted the JEDI transport for a while.

The only really complicated bits are making sure that

(i) transactions work (so all message exchanges with transactions go
to a single broker only) - though you might wish to have N
transactions on each of your N broker connections and

(ii) queue messages only go to one broker

(iii) topic messages go to all messages.

(iv) MessageAck only go to the broker which delivered the Message.

So the only real difference between FanOut transport is choosing the
broker(s) that an outbound Command object goes to. Fancy taking a stab
at it? :)

-- 
James
-------
http://macstrac.blogspot.com/

Mime
View raw message