activemq-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Adrian Trenaman (JIRA)" <>
Subject [jira] Created: (AMQ-2875) Create a <treeConnector> for ActiveMQ to provide better support for hierarchical broker topologies
Date Mon, 23 Aug 2010 11:50:47 GMT
Create a <treeConnector> for ActiveMQ to provide better support for hierarchical broker

                 Key: AMQ-2875
             Project: ActiveMQ
          Issue Type: New Feature
            Reporter: Adrian Trenaman

The ActiveMQ network connector is excellent at facilitating self-organizing networks of ActiveMQ
brokers with dynamically discovered routing, and can be used in situations where network availability
is sporadic and not guaranteed. Network connectors can be used along with master-slave pairs
to create a distributed 'messaging fabric'. The network connector can also accommodate the
creation of hierarchical networks, however, I believe there is a good argument for treating
tree topologies as a special case in their own right. In this note, I'm going to describe
why trees should be treated as a special case, and describe what a treeConnector might look

Hierarchical (or 'tree') networks are a special case of the abstract notion of a network;
they have numerous applications in the real world in cross-geography, wide-area deployments.
Consider a retail organization with 1,000's of stores, who wish to send and receive information
to and from head-quarters (HQ) and the stores: you can envisage a hierarchy of ActiveMQ brokers:
one for HQ, one for each of the regions (e.g. Ireland, UK, France, Germany), and then a broker
in each store/outlet in each of the regions. When HQ wants to send a message to a store, it
should be able to write a message to the store's queue and have it dynamically routed to the
store via the regional broker. Likewise, if a store wants to send a message to HQ, it should
be able to write to a local queue in the store and have the message dynamically routed to

Right now in ActiveMQ, there are a number of issues with using a network connector to achieve
a tree topology: 

* Spillage of consumer advisories. When a consumer connects to a store, an advisory message
is generated and sent to all brokers within the networkTTL range. These advisories don't just
travel up the tree to the top (HQ): they can also spill back down the tree to peers who are
simply not interested. We have seen this in a network with 1,000 brokers connected to a regional
broker, and a networkTTL of two: the advisory gets sent to the regional broker, HQ, *and*
the 999 peers of the broker. 

* Resource wastage (as a consequence of spillage). When a consumer advisory from a store broker
(say, broker 'A') reaches another broker within the networkTTL (say, broker 'B'), then 'B'
creates a subscription for the destination (to do forwarding) *and* can result in the allocation
of a thread for this destination. So, it's possible that Broker B will allocate a thread for
the destination 'BrokerA.Incoming', despite the fact that it will never receive or send to
this queue. If there are only a small number of peers at this level then this is not a problem;
however, if there are many peers (as in the case of 1000 stores per region) then this will
be noticeable.

* Sensitivity to networkTTL. In order to minimize spillage and resource wastage, you need
to se the networkTTL on each broker to the distance between the broker and the root - in this
example, 2. However, a later reconfiguration of the network could result in this setting being
too low (which means messages won't go the distance) or too high (which means you get inadvertent

These issues can be addressed if we create a new tree connector for ActiveMQ. 

1) Brokers would identify themselves as 'leaf', 'branch' or 'root' nodes. There should only
ever be one root. If a broker has not been identified as either 'leaf' or 'branch' then it
should consider itself to be the root.

2) Tree connectors would be configured using a 'many to one' approach. Leaf brokers would
configure a tree connector to their branch. Branch brokers would configure a tree connector
to the root. The root shouldn't need to configure anything, except of course a transport listener.

So, a branch broker for the 'UK' region might have the following  (to connect to headquarters):

	<treeConnector nodeType="branch" uri=""tcp://"/>

And, a leaf broker at a store or outlet in the UK might have the following (to connect to
the 'UK' region'): 

	<treeConnector nodeType="leaf" uri=""tcp://"/>

3) If a consumer connects to a leaf node, then the consumer advisory travels up the tree,
all the way to root: the concept of networkTTL is ignored. If this advisory travels through
a 'branch' then the branch delegates the advisory upwards. Branches do not pass advisory messages
from children to their other child nodes.

4) If a consumer connects to a branch node, then the consumer advisory travels up the tree.
Additionally, the branch broker will send the advisory downwards to its children.

5) If a consumer connects to the root, then the consumer advisory is filtered downwards to
all children.

In this way, we reduce spillage of advisory messages, and ensure that trees can self organize
without too much intervention or worrying about the network time-to-live.


This message is automatically generated by JIRA.
You can reply to this email to add a comment to the issue online.

View raw message