activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Joe Fernandez <joe.fernan...@ttmsolutions.com>
Subject Re: sender side manual load balancing
Date Wed, 16 Jun 2010 14:59:18 GMT

If you want the brokers to be independent of one another (i.e., they're not
interconnected), then don't include networkConnectors (forwarding bridges)
in their configurations.  

On the clients side, I think the discovery agent/connector should give you
what you want.

http://activemq.apache.org/discovery-transport-reference.html
   
Joe
http://www.ttmsolutions.com
ActiveMQ Ref Guide - http://bit.ly/AMQRefGuide



bacevt wrote:
> 
> Hi,
> 
> I am trying to find the best scalability setup regarding my JMS scenario,
> and I would like to ask you to help me to do that. The scenario is very
> simple – I have a set of producers, which send thousands of persistent
> messages on a queue for a several seconds, as a consequence of some UI
> activity. I also have a set of consumers, responsible for message
> processing. The order of message processing is not important – each
> message is self contained, and its processing is completely independent of
> the other’s messages processing. The processing is a very complex, and
> typically it takes one or several minutes to complete the message
> processing.
> 
> In order to improve my application scalability, I would like to use
> several independent brokers and delegate the decision to the sender
> application which broker would to send a message to.
> I don’t think to use a single broker, /despite of the fact that this is a
> highly recommended solution/ because I don’t want to have a single point
> of failure. Network of brokers is a useful option for me, but I am afraid
> that this will introduce addition network overhead. This “sender side
> manual load balancing” combines the best of the both worlds – no single
> point of failure and no cluster communication overhead. 
> 
> Probably this scenario is very similar to the jedi-transport request: 
> https://issues.apache.org/activemq/browse/AMQ-816
> However, jedi-transport is still unavailable, so it is not a real option. 
> 
> In order to delegate the decision to choose a broker to which to send a
> message, the sender application should be able to know the list of all
> available brokers. One possible solution is to use a hard coded list,
> however a more advanced solution will be able to detect dynamically
> available brokers. The idea is to implement discovery listener on the
> client side and to listen for discovery events and to maintain a set of
> available brokers. The sender application will be able to send a message
> using round robin load balancing between brokers.
> 
> On the other hand I need to use broker’s discovery mechanism, but with
> some modification – I expect brokers to expose themselves, but not to
> establish a network of brokers between them. I checked the code, and
> regarding my findings the only possible way to do that is to setup each
> broker in different group and to share one and the same multicast address.
> For example:
> 
> Broker1 -> discoveryUri="multicast://default?group=broker1"
> Broker2 -> discoveryUri="multicast://default?group=broker2"
> …
> BrokerN -> discoveryUri="multicast://default?group=brokerN"
> 
> This setup will challenge all brokers to advertise themselves and at the
> same time each broker belongs to its own group. I don’t expect additional
> network traffic due to clustering, except the packets sent by discovery
> agents. This will provide a solution for membership detection to the
> sender application – it simply listens on the same multicast address and
> analyses the incoming notifications. For example:
> 
> received: broker1.ActiveMQ-4.alive.%localhost%tcp://mymachine:61616
> received: broker2.ActiveMQ-4.alive.%localhost%tcp://mymachine:61618
> received: broker1.ActiveMQ-4.alive.%localhost%tcp://mymachine:61616
> received: broker1.ActiveMQ-4.dead.%localhost%tcp://mymachine:61616
> received: broker2.ActiveMQ-4.alive.%localhost%tcp://mymachine:61618
> received: broker2.ActiveMQ-4.dead.%localhost%tcp://mymachine:61618
> 
> Probably this is very tricky and risky solution, since the client side
> application should analyze a data packets, used only internally. I need to
> re-implement the logic of MulticastDiscoveryListener with the main aim to
> implement DiscoveryListener. The implementation of this listener will
> receive notification about service start/stop and will maintain a set of
> the available subscribers.
> The sender application will be able to discover dynamically all available
> brokers and to use some load balancing strategy to utilize them.
> 
> Do you think that my approach is siutable?
> Do you see any serious downfalls in my suggestion? 
> Any help will be highly appreciated.
> 
> 
> Greetings
> 

-- 
View this message in context: http://old.nabble.com/sender-side-manual-load-balancing-tp28903218p28903840.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Mime
View raw message