activemq-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From bacevt <bac...@gmail.com>
Subject Re: sender side manual load balancing
Date Wed, 23 Jun 2010 11:19:56 GMT

Hello Joe,
Thank you for your reply, this is just what I needed.

Best regards, bacevt



Joe Fernandez wrote:
> 
> If you want the brokers to be independent of one another (i.e., they're
> not interconnected), then don't include networkConnectors (forwarding
> bridges) in their configurations.  
> 
> On the clients side, I think the discovery agent/connector should give you
> what you want.
> 
> http://activemq.apache.org/discovery-transport-reference.html
>    
> Joe
> http://www.ttmsolutions.com
> ActiveMQ Ref Guide - http://bit.ly/AMQRefGuide
> 
> 
> 
> bacevt wrote:
>> 
>> Hi,
>> 
>> I am trying to find the best scalability setup regarding my JMS scenario,
>> and I would like to ask you to help me to do that. The scenario is very
>> simple – I have a set of producers, which send thousands of persistent
>> messages on a queue for a several seconds, as a consequence of some UI
>> activity. I also have a set of consumers, responsible for message
>> processing. The order of message processing is not important – each
>> message is self contained, and its processing is completely independent
>> of the other’s messages processing. The processing is a very complex, and
>> typically it takes one or several minutes to complete the message
>> processing.
>> 
>> In order to improve my application scalability, I would like to use
>> several independent brokers and delegate the decision to the sender
>> application which broker would to send a message to.
>> I don’t think to use a single broker, /despite of the fact that this is a
>> highly recommended solution/ because I don’t want to have a single point
>> of failure. Network of brokers is a useful option for me, but I am afraid
>> that this will introduce addition network overhead. This “sender side
>> manual load balancing” combines the best of the both worlds – no single
>> point of failure and no cluster communication overhead. 
>> 
>> Probably this scenario is very similar to the jedi-transport request: 
>> https://issues.apache.org/activemq/browse/AMQ-816
>> However, jedi-transport is still unavailable, so it is not a real option. 
>> 
>> In order to delegate the decision to choose a broker to which to send a
>> message, the sender application should be able to know the list of all
>> available brokers. One possible solution is to use a hard coded list,
>> however a more advanced solution will be able to detect dynamically
>> available brokers. The idea is to implement discovery listener on the
>> client side and to listen for discovery events and to maintain a set of
>> available brokers. The sender application will be able to send a message
>> using round robin load balancing between brokers.
>> 
>> On the other hand I need to use broker’s discovery mechanism, but with
>> some modification – I expect brokers to expose themselves, but not to
>> establish a network of brokers between them. I checked the code, and
>> regarding my findings the only possible way to do that is to setup each
>> broker in different group and to share one and the same multicast
>> address. For example:
>> 
>> Broker1 -> discoveryUri="multicast://default?group=broker1"
>> Broker2 -> discoveryUri="multicast://default?group=broker2"
>> …
>> BrokerN -> discoveryUri="multicast://default?group=brokerN"
>> 
>> This setup will challenge all brokers to advertise themselves and at the
>> same time each broker belongs to its own group. I don’t expect additional
>> network traffic due to clustering, except the packets sent by discovery
>> agents. This will provide a solution for membership detection to the
>> sender application – it simply listens on the same multicast address and
>> analyses the incoming notifications. For example:
>> 
>> received: broker1.ActiveMQ-4.alive.%localhost%tcp://mymachine:61616
>> received: broker2.ActiveMQ-4.alive.%localhost%tcp://mymachine:61618
>> received: broker1.ActiveMQ-4.alive.%localhost%tcp://mymachine:61616
>> received: broker1.ActiveMQ-4.dead.%localhost%tcp://mymachine:61616
>> received: broker2.ActiveMQ-4.alive.%localhost%tcp://mymachine:61618
>> received: broker2.ActiveMQ-4.dead.%localhost%tcp://mymachine:61618
>> 
>> Probably this is very tricky and risky solution, since the client side
>> application should analyze a data packets, used only internally. I need
>> to re-implement the logic of MulticastDiscoveryListener with the main aim
>> to implement DiscoveryListener. The implementation of this listener will
>> receive notification about service start/stop and will maintain a set of
>> the available subscribers.
>> The sender application will be able to discover dynamically all available
>> brokers and to use some load balancing strategy to utilize them.
>> 
>> Do you think that my approach is siutable?
>> Do you see any serious downfalls in my suggestion? 
>> Any help will be highly appreciated.
>> 
>> 
>> Greetings
>> 
> 
> 

-- 
View this message in context: http://old.nabble.com/sender-side-manual-load-balancing-tp28903218p28970514.html
Sent from the ActiveMQ - User mailing list archive at Nabble.com.


Mime
View raw message