activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Tully <gary.tu...@gmail.com>
Subject Re: Load Balancing Producer Messages from the JMS Client side
Date Wed, 19 Sep 2012 11:20:02 GMT
there is no easy way to do this today out of the box.

it has be discussed many times, and there is even a name, ninja client.

Something that has all the smarts about brokers and their load and can
return a connection with some guarantees w.r.t to throughput etc.

The statistics broker makes broker stats available over jms, so a
smarter pool could could keep track of numer of connections or some
simple metric like that.
Or even periodically do a message roundtrip to validate some threshold.

But none of this exists today so it needs to be build, it would make a
neat sub module.

I think one bit that may be missing is a way to get a handle on the
dynamic list of brokers from a static network, I am not sure that is
propagated past the failover transport at the moment. That would need
to be validated. see:
http://sensatic.net/activemq/new-activemq-failover-and-clustering-goodies.html

I could see a 'ninja' connection pool as a great start point.

On 18 September 2012 23:43, wjmcdonald <william.mcdonald@transcentra.com> wrote:
> If I understand how this works, there isn't a way to have a JMS producer send
> messages in a load balanced way directly to multiple brokers (random,
> round-robin, algorithm based upon feedback from brokers, etc) unless I want
> to make my own thread pool that opens connections to a URI list, correct?
>
> So currently, I can use a connection pool to create connections to ONE
> broker.  That broker can be chosen for me from a list via eg.
> discovery:(fabric:groupName) or discovery:(failover:(tcp://localhost:123,
> tcp://remote:321)), but once the JMS client selects a URL, as long as its
> still connected, it will only create connections and send messages to that
> broker, correct?
>
> So with 2 brokers and 1 client, where the load on the first broker gets
> large, yet the second broker could handle traffic from the client, but won't
> because the client only targets the first broker, right?  (unless I roll my
> own client impl using threading for example with connections to both
> brokers).  I know in this scenario, one solution is to simply network the
> brokers, then the first broker could off-load some messages to consumer(s)
> on the second broker, but I'm focusing on the problem of overloading the
> first broker with incoming traffic.  It would be nice if some of that
> traffic didn't even hit the first broker just to be load balanced.
>
> Initially I thought, it would be nice if these discovery methods favored the
> most efficient URL (like those that are local to the client vs remote), but
> then realized that there may be other factors that would override this
> simple algorithm.  For example,
> 1.  If the local brokers are really overloaded and a remote one could easily
> take the work, it might be cheaper to pay the networking cost to ship it
> there (and possibly back) than to throw more load on the local brokers.
> 2. There might be a quality of service that the available local broker is
> trying to guarantee and won't take on the work even though its cheaper for
> the current message that the client is trying to send out (think of the
> 'elite' line at the airport).
> 3. The remote broker might have the queue capacity that the local one
> doesn't, etc.
>
> (I noticed the fail-back of the priorityBackup in activemq, but didn't know
> if/how this might work with fabric.)
>
> It seems like there are 2 things that are important to making the most
> efficient use of the broker connections.
> 1. a (dynamicaly updating) list of brokers
> 2. an information source on how they are doing, the costs associated with
> routing and intelligent algorithm(s) agent(s) to orchestrate the traffic to
> all the brokers in the list to accomplish specific goal(s) like throughput,
> QOS, etc.
>
> Is there any way to do this today?  If so, how?  Would the solution work in
> Camel?
>
> TIA
>
>
>
>
>
> --
> View this message in context: http://activemq.2283324.n4.nabble.com/Load-Balancing-Producer-Messages-from-the-JMS-Client-side-tp4656611.html
> Sent from the ActiveMQ - User mailing list archive at Nabble.com.



-- 
http://fusesource.com
http://blog.garytully.com

Mime
View raw message