activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From wjmcdonald <william.mcdon...@transcentra.com>
Subject Load Balancing Producer Messages from the JMS Client side
Date Tue, 18 Sep 2012 22:43:12 GMT
If I understand how this works, there isn't a way to have a JMS producer send
messages in a load balanced way directly to multiple brokers (random,
round-robin, algorithm based upon feedback from brokers, etc) unless I want
to make my own thread pool that opens connections to a URI list, correct? 

So currently, I can use a connection pool to create connections to ONE
broker.  That broker can be chosen for me from a list via eg.
discovery:(fabric:groupName) or discovery:(failover:(tcp://localhost:123,
tcp://remote:321)), but once the JMS client selects a URL, as long as its
still connected, it will only create connections and send messages to that
broker, correct?

So with 2 brokers and 1 client, where the load on the first broker gets
large, yet the second broker could handle traffic from the client, but won't
because the client only targets the first broker, right?  (unless I roll my
own client impl using threading for example with connections to both
brokers).  I know in this scenario, one solution is to simply network the
brokers, then the first broker could off-load some messages to consumer(s)
on the second broker, but I'm focusing on the problem of overloading the
first broker with incoming traffic.  It would be nice if some of that
traffic didn't even hit the first broker just to be load balanced.

Initially I thought, it would be nice if these discovery methods favored the
most efficient URL (like those that are local to the client vs remote), but
then realized that there may be other factors that would override this
simple algorithm.  For example,
1.  If the local brokers are really overloaded and a remote one could easily
take the work, it might be cheaper to pay the networking cost to ship it
there (and possibly back) than to throw more load on the local brokers.  
2. There might be a quality of service that the available local broker is
trying to guarantee and won't take on the work even though its cheaper for
the current message that the client is trying to send out (think of the
'elite' line at the airport).  
3. The remote broker might have the queue capacity that the local one
doesn't, etc.

(I noticed the fail-back of the priorityBackup in activemq, but didn't know
if/how this might work with fabric.)

It seems like there are 2 things that are important to making the most
efficient use of the broker connections.
1. a (dynamicaly updating) list of brokers 
2. an information source on how they are doing, the costs associated with
routing and intelligent algorithm(s) agent(s) to orchestrate the traffic to
all the brokers in the list to accomplish specific goal(s) like throughput,
QOS, etc.

Is there any way to do this today?  If so, how?  Would the solution work in
Camel?

TIA





--
View this message in context: http://activemq.2283324.n4.nabble.com/Load-Balancing-Producer-Messages-from-the-JMS-Client-side-tp4656611.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.

Mime
View raw message