Qpid Dispatch Router Book

1. Introduction

1.1. Overview

The Dispatch router is an AMQP message router that provides +advanced interconnect capabilities. It allows flexible routing of +messages between any AMQP-enabled endpoints, whether they be clients, +servers, brokers or any other entity that can send or receive standard +AMQP messages.

A messaging client can make a single AMQP connection into a messaging +bus built of Dispatch routers and, over that connection, exchange +messages with one or more message brokers, and at the same time exchange +messages directly with other endpoints without involving a broker at +all.

The router is an intermediary for messages but it is not a broker. It +does not take responsibility for messages. It will, however, propagate +settlement and disposition across a network such that delivery +guarantees are met. In other words: the router network will deliver the +message, possibly via several intermediate routers, and it will route +the acknowledgement of that message by the ultimate receiver back across +the same path. This means that responsibility for the message is +transfered from the original sender to the ultimate receiver as if +they were directly connected. However this is done via a flexible +network that allows highly configurable routing of the message +transparent to both sender and receiver.

There are some patterns where this enables "brokerless messaging" +approaches that are preferable to brokered approaches. In other cases a +broker is essential (in particular where you need the separation of +responsibility and/or the buffering provided by store-and-forward) but a +dispatch network can still be useful to tie brokers and clients together +into patterns that are difficult with a single broker.

For a "brokerless" example, consider the common brokered implementation +of the request-response pattern, a client puts a request on a queue and +then waits for a reply on another queue. In this case the broker can be +a hindrance - the client may want to know immediatly if there is nobody +to serve the request, but typically it can only wait for a timeout to +discover this. With a dispatch network, the client can be informed +immediately if its message cannot be delivered because nobody is +listening. When the client receives acknowledgement of the request it +knows not just that it is sitting on a queue, but that it has actually +been received by the server.

For an exampe of using dispatch to enhance the use of brokers, consider +using an array of brokers to implement a scalable distributed work +queue. A dispatch network can make this appear as a single queue, with +senders publishing to a single address and receivers subscribing to a +single address. The dispatch network can distribute work to any broker +in the array and collect work from any broker for any receiver. Brokers +can be shut down or added without affecting clients. This elegantly +solves the common difficulty of "stuck messages" when implementing this +pattern with brokers alone. If a receiver is connected to a broker that +has no messages, but there are messages on another broker, you have to +somehow transfer them or leave them "stuck". With a dispatch network, +all the receivers are connected to all the brokers. If there is a +message anywhere it can be delivered to any receiver.

The router is meant to be deployed in topologies of multiple routers, +preferably with redundant paths. It uses link-state routing protocols +and algorithms (similar to OSPF or IS-IS from the networking world) to +calculate the best path from every point to every other point and to +recover quickly from failures. It does not need to use clustering for +high availability; rather, it relies on redundant paths to provide +continued connectivity in the face of system or network failure. Because +it never takes responsibility for messages it is effectively stateless. +Messages not delivered to their final destination will not be +acknowledged to the sender and therefore the sender can re-send such +messages if it is disconnected from the network.

1.2. Benefits

Simplifies connectivity

+
+An endpoint can do all of its messaging through a single transport +connection +
+
+
+Avoid opening holes in firewalls for incoming connections +
+

Provides messaging connectivity where there is no TCP/IP connectivity

+
+A server or broker can be in a private IP network (behind a NAT +firewall) and be accessible by messaging endpoints in other networks +(learn more). +
+

Simplifies reliability

+
+Reliability and availability are provided using redundant topology, +not server clustering +
+
+
+Reliable end-to-end messaging without persistent stores +
+
+
+Use a message broker only when you need store-and-forward semantics +
+

1.3. Features

+
+Can be deployed stand-alone or in a network of routers +
+
- +
  +Supports arbitrary network topology - no restrictions on redundancy +
  +
  - +
    +Automatic route computation - adjusts quickly to changes in topology +
    +
  +
+
+
+Provides remote access to brokers or other AMQP servers +
+
+
+Security +
+

2. Using Qpid Dispatch

2.1. Configuration

The default configuration file is installed in +install-prefix/etc/qpid-dispatch/qdrouterd.conf. This configuration file will +cause the router to run in standalone mode, listening on the standard +AMQP port (5672). Dispatch Router looks for the configuration file in +the installed location by default. If you wish to use a different path, +the "-c" command line option will instruct Dispatch Router as to which +configuration to load.

To run the router, invoke the executable: qdrouterd [-c my-config-file]

For more details of the configuration file see the qdrouterd.conf(5) man +page.

2.2. Tools

2.2.1. qdstat

qdstat is a command line tool that lets you view the status of a +Dispatch Router. The following options are useful for seeing what the +router is doing:

+ +++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Option	Description
-l	Print a list of AMQP links attached to the router. Links are +unidirectional. Outgoing links are usually associated with a +subscription address. The tool distinguishes between endpoint links +and router links. Endpoint links are attached to clients using the +router. Router links are attached to other routers in a network of +routbers.
-a	Print a list of addresses known to the router.
-n	Print a list of known routers in the network.
-c	Print a list of connections to the router.
--autolinks	Print a list of configured auto-links.
--linkroutes	Print a list of configures link-routes.

For complete details see the qdstat(8) man page and the output of +qdstat --help.

2.2.2. qdmanage

qdmanage is a general-purpose AMQP management client that allows you +to not only view but modify the configuration of a running dispatch +router.

For example you can query all the connection entities in the router:

$ qdmanage query --type connection

To enable logging debug and higher level messages by default:

$ qdmanage update log/DEFAULT enable=debug+

In fact, everything that can be configured in the configuration file can +also be created in a running router via management. For example to +create a new listener in a running router:

$ qdmanage create type=listener port=5555

Now you can connect to port 5555, for exampple:

$ qdmanage query -b localhost:5555 --type listener

For complete details see the qdmanage(8) man page and the output of +qdmanage --help. Also for details of what can be configured see the +qdrouterd.conf(5) man page.

2.3. Basic Usage and Examples

2.3.1. Standalone and Interior Modes

The router can operate stand-alone or as a node in a network of routers. +The mode is configured in the router section of the configuration +file. In stand-alone mode, the router does not attempt to collaborate +with any other routers and only routes messages among directly connected +endpoints.

If your router is running in stand-alone mode, qdstat -a will look +like the following:

$ qdstat -a
+Router Addresses
+  class   addr                   phs  distrib  in-proc  local  remote  cntnr  in  out  thru  to-proc  from-proc
+  ===============================================================================================================
+  local   $_management_internal       closest  1        0      0       0      0   0    0     0        0
+  local   $displayname                closest  1        0      0       0      0   0    0     0        0
+  mobile  $management            0    closest  1        0      0       0      1   0    0     1        0
+  local   $management                 closest  1        0      0       0      0   0    0     0        0
+  local   temp.1GThUllfR7N+BDP        closest  0        1      0       0      0   0    0     0        0

Note that there are a number of known addresses. $management is the +address of the router’s embedded management agent. +temp.1GThUllfR7N+BDP is the temporary reply-to address of the qdstat +client making requests to the agent.

If you change the mode to interior and restart the processs, the same +command will yield additional addresses which are used for inter-router +communication:

$ qdstat -a
+Router Addresses
+  class   addr                   phs  distrib    in-proc  local  remote  cntnr  in  out  thru  to-proc  from-proc
+  =================================================================================================================
+  local   $_management_internal       closest    1        0      0       0      0   0    0     0        0
+  local   $displayname                closest    1        0      0       0      0   0    0     0        0
+  mobile  $management            0    closest    1        0      0       0      1   0    0     1        0
+  local   $management                 closest    1        0      0       0      0   0    0     0        0
+  local   qdhello                     flood      1        0      0       0      0   0    0     0        10
+  local   qdrouter                    flood      1        0      0       0      0   0    0     0        0
+  topo    qdrouter                    flood      1        0      0       0      0   0    0     0        1
+  local   qdrouter.ma                 multicast  1        0      0       0      0   0    0     0        0
+  topo    qdrouter.ma                 multicast  1        0      0       0      0   0    0     0        0
+  local   temp.wfx54+zf+YWQF3T        closest    0        1      0       0      0   0    0     0        0

2.3.2. Mobile Subscribers

The term "mobile subscriber" simply refers to the fact that a client may +connect to the router and subscribe to an address to receive messages +sent to that address. No matter where in the network the subscriber +attaches, the messages will be routed to the appropriate destination.

To illustrate a subscription on a stand-alone router, you can use the +examples that are provided with Qpid Proton. Using the simple_recv.py +example receiver:

$ python ./simple_recv.py -a 127.0.0.1/my-address

This command creates a receiving link subscribed to the specified +address. To verify the subscription:

$ qdstat -a
+Router Addresses
+  class   addr                   phs  distrib  in-proc  local  remote  cntnr  in  out  thru  to-proc  from-proc
+  ===============================================================================================================
+  local   $_management_internal       closest  1        0      0       0      0   0    0     0        0
+  local   $displayname                closest  1        0      0       0      0   0    0     0        0
+  mobile  $management            0    closest  1        0      0       0      2   0    0     2        0
+  local   $management                 closest  1        0      0       0      0   0    0     0        0
+  mobile  my-address             0    closest  0        1      0       0      0   0    0     0        0
+  local   temp.75_d2X23x_KOT51        closest  0        1      0       0      0   0    0     0        0

You can then, in a separate command window, run a sender to produce +messages to that address:

$ python ./simple_send.py -a 127.0.0.1/my-address

2.3.3. Dynamic Reply-To

Dynamic reply-to can be used to obtain a reply-to address that routes +back to a client’s receiving link regardless of how many hops it has to +take to get there. To illustrate this feature, see below a simple +program (written in C++ against the qpid::messaging API) that queries +the management agent of the attached router for a list of other known +routers' management addresses.

#include <qpid/messaging/Address.h>
+#include <qpid/messaging/Connection.h>
+#include <qpid/messaging/Message.h>
+#include <qpid/messaging/Receiver.h>
+#include <qpid/messaging/Sender.h>
+#include <qpid/messaging/Session.h>
+
+using namespace qpid::messaging;
+using namespace qpid::types;
+
+using std::stringstream;
+using std::string;
+
+int main() {
+    const char* url = "amqp:tcp:127.0.0.1:5672";
+    std::string connectionOptions = "{protocol:amqp1.0}";
+
+    Connection connection(url, connectionOptions);
+    connection.open();
+    Session session = connection.createSession();
+    Sender sender = session.createSender("mgmt");
+
+    // create reply receiver and get the reply-to address
+    Receiver receiver = session.createReceiver("#");
+    Address responseAddress = receiver.getAddress();
+
+    Message request;
+    request.setReplyTo(responseAddress);
+    request.setProperty("x-amqp-to", "amqp:/_local/$management");
+    request.setProperty("operation", "DISCOVER-MGMT-NODES");
+    request.setProperty("type", "org.amqp.management");
+    request.setProperty("name, "self");
+
+    sender.send(request);
+    Message response = receiver.fetch();
+    Variant content(response.getContentObject());
+    std::cout << "Response: " << content << std::endl << std::endl;
+
+    connection.close();
+}

The equivalent program written in Python against the Proton Messenger +API:

from proton import Messenger, Message
+
+def main():
+    host = "0.0.0.0:5672"
+
+    messenger = Messenger()
+    messenger.start()
+    messenger.route("amqp:/*", "amqp://%s/$1" % host)
+    reply_subscription = messenger.subscribe("amqp:/#")
+    reply_address = reply_subscription.address
+
+    request  = Message()
+    response = Message()
+
+    request.address = "amqp:/_local/$management"
+    request.reply_to = reply_address
+    request.properties = {u'operation' : u'DISCOVER-MGMT-NODES',
+                          u'type'      : u'org.amqp.management',
+                          u'name'      : u'self'}
+
+    messenger.put(request)
+    messenger.send()
+    messenger.recv()
+    messenger.get(response)
+
+    print "Response: %r" % response.body
+
+    messenger.stop()
+
+main()

2.4. Link Routing

This feature was introduced in Qpid Dispatch 0.4. This feature was +significantly updated in Qpid Dispatch 0.6.

Link-routing is an alternative strategy for routing messages across a +network of routers. With the existing message-routing strategy, each +router makes a routing decision on a per-message basis when the message +is delivered. Link-routing is different because it makes routing +decisions when link-attach frames arrive. A link is effectively chained +across the network of routers from the establishing node to the +destination node. Once the link is established, the transfer of message +deliveries, flow frames, and dispositions is performed across the routed +link.

The main benefit to link-routing is that endpoints can use the full link +protocol to interact with other endpoints in far-flung parts of the +network. For example, a client can establish a receiver across the +network to a queue on a remote broker and use link credit to control +the flow of messages from the broker. Similarly, a receiver can +establish a link to a topic on a remote broker using a server-side +filter.

Why would one want to do this? One reason is to provide client +isolation. A network like the following can be deployed:

                        Public Network
+                       +-----------------+
+                       |      +-----+    |
+                       | B1   | Rp  |    |
+                       |      +/--\-+    |
+                       |      /    \     |
+                       |     /      \    |
+                       +----/--------\---+
+                           /          \
+                          /            \
+                         /              \
+         Private Net A  /                \ Private Net B
+        +--------------/--+           +---\-------------+
+        |         +---/-+ |           | +--\--+         |
+        |  B2     | Ra  | |           | | Rb  |   C1    |
+        |         +-----+ |           | +-----+         |
+        |                 |           |                 |
+        |                 |           |                 |
+        +-----------------+           +-----------------+

The clients in Private Net B can be constrained (by firewall policy) +to only connect to the Router in their own network. Using +link-routing, these clients can access queues, topics, and other AMQP +services that are in the Public Network or even in Private Net A.

For example, The router Ra can be configured to expose queues in +broker B2 to the network. Client C1 can then establish a connection +to Rb, the local router, open a subscribing link to "b2.event-queue", +and receive messages stored on that queue in broker B2.

C1 is unable to create a TCP/IP connection to B1 because of its +isolation (and because B2 is itself in a private network). However, with +link routing, C1 can interact with B2 using the AMQP link protocol.

Note that in this case, neither C1 nor B2 have been modified in any way +and neither need be aware of the fact that there is a message-router +network between them.

2.4.1. Configuration

Starting with the configured topology shown above, how is link-routing +configured to support the example described above?

First, router Ra needs to be told how to make a connection to the broker +B2:

connector {
+    name: broker
+    role: route-container
+    host: <B2-url>
+    port: <B2-port>
+    saslMechanisms: ANONYMOUS
+}

This route-container connector tells the router how to connect to an +external AMQP container when it is needed. The name "broker" will be +used later to refer to this connection.

Now, the router must be configured to route certain addresses to B2:

linkRoute {
+    prefix: b2
+    dir: in
+    connection: broker
+}
+
+linkRoute {
+    prefix: b2
+    dir: out
+    connection: broker
+}

The linkRoute tells router Ra that any sender or receiver that is +attached with a target or source (respectively) whos address begins with +"b2", should be routed to the broker B2 (via the route-container +connector).

Note that receiving and sending links are configured and routed +separately. This allows configuration of link routes for listeners only +or senders only. A direction of "in" matches client senders (i.e. links +that carry messages inbound to the router network). Direction "out" +matches client receivers.

Examples of addresses that "begin with b2" include:

+
+b2 +
+
+
+b2.queues +
+
+
+b2.queues.app1 +
+

When the route-container connector is configured, router Ra establishes +a connection to the broker. Once the connection is open, Ra tells the +other routers (Rp and Rb) that it is a valid destination for link-routes +to the "b2" prefix. This means that sender or receiver links attached to +Rb or Rp will be routed via the shortest path to Ra where they are then +routed outbound to the broker B2.

On Rp and Rb, it is advisable to add the identical configuration. It is +permissible for a linkRoute configuration to reference a connection that +does not exist.

This configuration tells the routers that link-routing is intended to be +available for targets and sources starting with "b2". This is important +because it is possible that B2 might be unavailable or shut off. If B2 +is unreachable, Ra will not advertize itself as a destination for "b2" +and the other routers might never know that "b2" was intended for +link-routing.

The above configuration allows Rb and Rp to reject attaches that should +be routed to B2 with an error message that indicates that there is no +route available to the destination.

2.5. Indirect Waypoints and Auto-Links

This feature was introduced in Qpid Dispatch 0.6. It is a significant +improvement on an earlier somewhat experimental feature called +Waypoints.

Auto-link is a feature of Qpid Dispatch Router that enables a router to +actively attach a link to a node on an external AMQP container. The +obvious application for this feature is to route messages through a +queue on a broker, but other applications are possible as well.

An auto-link manages the lifecycle of one AMQP link. If messages are to +be routed to and from a queue on a broker, then two auto-links are +needed: one for sending messages to the queue and another for receiving +messages from the queue. The container to which an auto-link attempts to +attach may be identified in one of two ways:

+
+The name of the connector/listener that resulted in the connection of +the container, or +
+
+
+The AMQP container-id of the remote container. +
+

2.5.1. Queue Waypoint Example

Here is an example configuration for routing messages deliveries through +a pair of queues on a broker:

connector {
+    name: broker
+    role: route-container
+    host: <hostname>
+    port: <port>
+    saslMechanisms: ANONYMOUS
+}
+
+address {
+    prefix: queue
+    waypoint: yes
+}
+
+autoLink {
+    addr: queue.first
+    dir: in
+    connection: broker
+}
+
+autoLink {
+    addr: queue.first
+    dir: out
+    connection: broker
+}
+
+autoLink {
+    addr: queue.second
+    dir: in
+    connection: broker
+}
+
+autoLink {
+    addr: queue.second
+    dir: out
+    connection: broker
+}

The address entity identifies a namespace queue. that will be used +for routing messages through queues via autolinks. The four autoLink entities +identify the head and tail of two queues on the broker that will be connected +via auto-links.

If there is no broker connected, the auto-links shall remain +inactive. This can be observed by using the qdstat tool:

$ qdstat --autolinks
+AutoLinks
+  addr          dir  phase  link  status    lastErr
+  ===================================================
+  queue.first   in   1            inactive
+  queue.first   out  0            inactive
+  queue.second  in   1            inactive
+  queue.second  out  0            inactive

If a broker comes online with a queue called queue.first, the +auto-links will attempt to activate:

$ qdstat --autolinks
+AutoLinks
+  addr          dir  phase  link  status  lastErr
+  ======================================================================
+  queue.first   in   1      6     active
+  queue.first   out  0      7     active
+  queue.second  in   1            failed  Node not found: queue.second
+  queue.second  out  0            failed  Node not found: queue.second

Note that two of the auto-links are in failed state because the queue +does not exist on the broker.

If we now use the Qpid Proton example application simple_send.py to send +three messages to queue.first via the router:

$ python simple_send.py -a 127.0.0.1/queue.first -m3
+all messages confirmed

and then look at the address statistics on the router:

$ qdstat -a
+Router Addresses
+  class   addr           phs  distrib   in-proc  local  remote  cntnr  in  out  thru  to-proc  from-proc
+  ========================================================================================================
+  mobile  queue.first    1    balanced  0        0      0       0      0   0    0     0        0
+  mobile  queue.first    0    balanced  0        1      0       0      3   3    0     0        0

we see that queue.first appears twice in the list of addresses. The +phs, or phase column shows that there are two phases for the +address. Phase 0 is for routing message deliveries from producers to +the tail of the queue (the out auto-link associated with the queue). +Phase 1 is for routing deliveries from the head of the queue to +subscribed consumers.

Note that three deliveries have been counted in the "in" and "out" +columns for phase 0. The "in" column represents the three messages +that arrived from simple_send.py and the out column represents the three +deliveries to the queue on the broker.

If we now use simple_recv.py to receive three messages from this address:

$ python simple_recv.py -a 127.0.0.1:5672/queue.first -m3
+{u'sequence': int32(1)}
+{u'sequence': int32(2)}
+{u'sequence': int32(3)}

We receive the three queued messages. Looking at the addresses again, we +see that phase 1 was used to deliver those messages from the queue to +the consumer.

$ qdstat -a
+Router Addresses
+  class   addr           phs  distrib   in-proc  local  remote  cntnr  in  out  thru  to-proc  from-proc
+  ========================================================================================================
+  mobile  queue.first    1    balanced  0        0      0       0      3   3    0     0        0
+  mobile  queue.first    0    balanced  0        1      0       0      3   3    0     0        0

Note that even in a multi-router network, and with multiple producers +and consumers for queue.first, all deliveries will be routed through +the queue on the connected broker.

2.5.2. Sharded Queue Example

Here is an extension of the above example to illustrate how Qpid +Dispatch Router can be used to create a distributed queue in which +multiple brokers share the message-queueing load.

connector {
+    name: broker1
+    role: route-container
+    host: <hostname>
+    port: <port>
+    saslMechanisms: ANONYMOUS
+}
+
+connector {
+    name: broker2
+    role: route-container
+    host: <hostname>
+    port: <port>
+    saslMechanisms: ANONYMOUS
+}
+
+address {
+    prefix: queue
+    waypoint: yes
+}
+
+autoLink {
+    addr: queue.first
+    dir: in
+    connection: broker1
+}
+
+autoLink {
+    addr: queue.first
+    dir: out
+    connection: broker1
+}
+
+autoLink {
+    addr: queue.first
+    dir: in
+    connection: broker2
+}
+
+autoLink {
+    addr: queue.first
+    dir: out
+    connection: broker2
+}

In the above configuration, there are two instances of queue.first on +brokers 1 and 2. Message traffic from producers to address queue.first +shall be balanced between the two instance and messages from the queues +shall be balanced across the collection of subscribers to the same +address.

2.5.3. Dynamically Adding Shards

Since configurable entities in the router can also be accessed via the +management protocol, we can remotely add a shard to the above example +using qdmanage:

qdmanage create --type org.apache.qpid.dispatch.connector host=<host> port=<port> name=broker3
+qdmanage create --type org.apache.qpid.dispatch.router.config.autoLink addr=queue.first dir=in connection=broker3
+qdmanage create --type org.apache.qpid.dispatch.router.config.autoLink addr=queue.first dir=out connection=broker3

3. Technical Details and Specifications

3.1. Client Compatibility

Dispatch Router should, in theory, work with any client that is +compatible with AMQP 1.0. The following clients have been tested:

+ +++ + + + + + + + + + + + + + + + + + + + +

Client	Notes
qpid::messaging	The Qpid messaging clients work with Dispatch Router +as long as they are configured to use the 1.0 version of the protocol. +To enable AMQP 1.0 in the C++ client, use the {protocol:amqp1.0} +connection option.
Proton Reactor	The Proton Reactor API is compatible with Dispatch +Router.
Proton Messenger	Messenger works with Dispatch Router.

3.2. Addressing

AMQP addresses are used to control the flow of messages across a network +of routers. Addresses are used in a number of different places in the +AMQP 1.0 protocol. They can be used in a specific message in the to +and reply-to fields of a message’s properties. They are also used +during the creation of links in the address field of a source or +a target.

Addresses designate various kinds of entities in a messaging network:

+
+Endpoint processes that consume data or offer a service +
+
+
+Topics that match multiple consumers to multiple producers +
+
+
+Entities within a messaging broker: +
+
- +
  +Queues +
  +
- +
  +Durable Topics +
  +
- +
  +Exchanges +
  +
+

The syntax of an AMQP address is opaque as far as the router network is +concerned. A syntactical structure may be used by the administrator that +creates addresses, but the router treats them as opaque strings. Routers +consider addresses to be mobile such that any address may be directly +connected to any router in a network and may move around the topology. +In cases where messages are broadcast to or balanced across multiple +consumers, an address may be connected to multiple routers in the +network.

Addresses have semantics associated with them. When an address is +created in the network, it is assigned a set of semantics (and access +rules) during a process called provisioning. The semantics of an address +control how routers behave when they see the address being used.

Address semantics include the following considerations:

+
+Routing pattern - direct, multicast, balanced +
+
+
+Undeliverable action - drop, hold and retry, redirect +
+
+
+Reliability - N destinations, etc. +
+

3.2.1. Routing patterns

Routing patterns constrain the paths that a message can take across a +network.

+ +++ + + + + + + + + + + + + + + + + + + + +

Pattern	Description
Direct	Direct routing allows for only one consumer to use an address +at a time. Messages (or links) follow the lowest cost path across the +network from the sender to the one receiver.
Multicast	Multicast routing allows multiple consumers to use the +same address at the same time. Messages are routed such that each +consumer receives a copy of the message.
Balanced	Balanced routing also allows multiple consumers to use the +same address. In this case, messages are routed to exactly one of the +consumers, and the network attempts to balance the traffic load across +the set of consumers using the same address.

3.2.2. Routing mechanisms

The fact that addresses can be used in different ways suggests that +message routing can be accomplished in different ways. Before going into +the specifics of the different routing mechanisms, it would be good to +first define what is meant by the term routing:

In a network built of multiple routers connected by connections (i.e., +nodes and edges in a graph), routing determines which connection to +use to send a message directly to its destination or one step closer to +its destination.

Each router serves as the terminus of a collection of incoming and +outgoing links. The links either connect directly to endpoints that +produce and consume messages, or they connect to other routers in the +network along previously established connections.

Message routing

Message routing occurs upon delivery of a message and is done based on +the address in the message’s to field.

When a delivery arrives on an incoming link, the router extracts the +address from the delivered message’s to field and looks the address up +in its routing table. The lookup results in zero or more outgoing links +onto which the message shall be resent.

+ +++ + + + + + + + + + + + + + + + +

Delivery	Handling
pre-settled	If the arriving delivery is pre-settled (i.e., fire and +forget), the incoming delivery shall be settled by the router, and the +outgoing deliveries shall also be pre-settled. In other words, the +pre-settled nature of the message delivery is propagated across the +network to the message’s destination.
unsettled	Unsettled delivery is also propagated across the network. +Because unsettled delivery records cannot be discarded, the router +tracks the incoming deliveries and keeps the association of the incoming +deliveries to the resulting outgoing deliveries. This kept association +allows the router to continue to propagate changes in delivery state +(settlement and disposition) back and forth along the path which the +message traveled.

Delivery

Handling

pre-settled

If the arriving delivery is pre-settled (i.e., fire and +forget), the incoming delivery shall be settled by the router, and the +outgoing deliveries shall also be pre-settled. In other words, the +pre-settled nature of the message delivery is propagated across the +network to the message’s destination.

unsettled

Unsettled delivery is also propagated across the network. +Because unsettled delivery records cannot be discarded, the router +tracks the incoming deliveries and keeps the association of the incoming +deliveries to the resulting outgoing deliveries. This kept association +allows the router to continue to propagate changes in delivery state +(settlement and disposition) back and forth along the path which the +message traveled.

3.3. AMQP Mapping

Dispatch Router is an AMQP router and as such, it provides extensions, +code-points, and semantics for routing over AMQP. This page documents the +details of Dispatch Router’s use of AMQP.

3.3.1. Message Annotations

The following Message Annotation fields are defined by Dispatch Router:

+ ++++ + + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Field	Type	Description
x-opt-qd.ingress	string	The identity of the ingress router for a +message-routed message. The ingress router is the first router +encountered by a transiting message. The router will, if this field is +present, leave it unaltered. If the field is not present, the router +shall insert the field with its own identity.
x-opt-qd.trace	list of string	The list of routers through which this +message-routed message has transited. If this field is not present, the +router shall do nothing. If the field is present, the router shall +append its own identity to the end of the list.
x-opt-qd.to	string	To-Override for message-routed messages. If this +field is present, the address in this field shall be used for routing in +lieu of the to field in the message properties. A router may append, +remove, or modify this annotation field depending on the policy in place +for routing the message.
x-opt-qd.phase	integer	The address-phase, if not zero, for messages +flowing between routers.

3.3.2. Source/Target Capabilities

The following Capability values are used in Sources and Targets.

+ +++ + + + + + + + + + + + + + + + +

Capability	Description
qd.router	This capability is added to sources and targets that are +used for inter-router message exchange. This capability denotes a link +used for router-control messages flowing between routers.
qd.router-data	This capability is added to sources and targets that +are used for inter-router message exchange. This capability denotes a +link used for user messages being message-routed across an inter-router +connection.

3.3.3. Dynamic-Node-Properties

The following dynamic-node-properties are used by Dispatch in Sources.

+ +++ + + + + + + + + + + + +

Property	Description
x-opt-qd.address	The node address describing the destination desired +for a dynamic source. If this is absent, the router will terminate any +dynamic receivers. If this address is present, the router will use the +address to route the dynamic link attach to the proper destination +container.

3.3.4. Addresses and Address Formats

The following AMQP addresses and address patterns are used within +Dispatch Router.

Address Patterns

+ +++ + + + + + + + + + + + + + + + + + + + +

Pattern Description

Pattern	Description
`_local/<addr>`	An address that references a locally attached endpoint. +Messages using this address pattern shall not be routed over more than +one link.
`_topo/0/<router>/<addr>`	An address that references an endpoint attached to a specific router +node in the network topology. Messages with addresses that follow this +pattern shall be routed along the shortest path to the specified router. +Note that addresses of this form are a-priori routable in that the +address itself contains enough information to route the message to its +destination. + The 0 component immediately preceding the router-id is a placeholder +for an area which may be used in the future if area routing is +implemented.
`<addr>`	A mobile address. An address of this format represents an +endpoint or a set of distinct endpoints that are attached to the network +in arbitrary locations. It is the responsibility of the router network +to determine which router nodes are valid destinations for mobile +addresses.

_local/<addr>

An address that references a locally attached endpoint. +Messages using this address pattern shall not be routed over more than +one link.

_topo/0/<router>/<addr>

An address that references an endpoint attached to a specific router +node in the network topology. Messages with addresses that follow this +pattern shall be routed along the shortest path to the specified router. +Note that addresses of this form are a-priori routable in that the +address itself contains enough information to route the message to its +destination.

The 0 component immediately preceding the router-id is a placeholder +for an area which may be used in the future if area routing is +implemented.

<addr>

A mobile address. An address of this format represents an +endpoint or a set of distinct endpoints that are attached to the network +in arbitrary locations. It is the responsibility of the router network +to determine which router nodes are valid destinations for mobile +addresses.

Supported Addresses

+ +++ + + + + + + + + + + + + + + + + + + + + + + + + + + + +

Address	Description
`$management`	The management agent on the attached router/container. +This address would be used by an endpoint that is a management +client/console/tool wishing to access management data from the attached +container.
`_topo/0/Router.E/$management`	The management agent at Router.E in area +0. This address would be used by a management client wishing to access +management data from a specific container that is reachable within the +network.
`_local/qdhello`	The router entity in each of the connected routers. +This address is used to communicate with neighbor routers and is +exclusively for the HELLO discovery protocol.
`_local/qdrouter`	The router entity in each of the connected routers. +This address is used by a router to communicate with other routers in +the network.
`_topo/0/Router.E/qdrouter`	The router entity at the specifically +indicated router. This address form is used by a router to communicate +with a specific router that may or may not be a neighbor.

3.3.5. Implementation of the AMQP Management Specification

Qpid Dispatch is manageable remotely via AMQP. It is compliant with the +emerging AMQP Management specification (draft 9).

Differences from the specification:

+
+The name attribute is not required when an entity is created. If + not supplied it will be set to the same value as the system-generated + "identity" attribute. Otherwise it is treated as per the standard. +
+
+
+The REGISTER and DEREGISTER operations are not implemented. The router + automatically discovers peer routers via the router network and makes + their management addresses available via the standard GET-MGMT-NODES + operation. += Management Schema +
+

This chapter documents the set of management entity types that define +configuration and management of a Dispatch Router. A management entity type has +a set of attributes that can be read, some attributes can also be +updated. Some entity types also support operations that can be called.

All management entity types have the following standard attributes:

+type +: +
+ The fully qualified type of the entity, + e.g. org.apache.qpid.dispatch.router. This document uses the short name + without the org.apache.qpid.dispatch prefix e.g. router. The dispatch + tools will accept the short or long name. +
+
+name +: +
+ A user-generated identity for the entity. This can be used in other entities + that need to refer to the named entity. +
+
+identity +: +
+ A system-generated identity of the entity. It includes + the short type name and some identifying information. E.g. log/AGENT or + listener/localhost:amqp +
+

There are two main categories of management entity type.

+Configuration Entities +: +
+ Parameters that can be set in the configuration file + (see qdrouterd.conf(5) man page) or set at run-time with the qdmanage(8) + tool. +
+
+Operational Entities +: +
+ Run-time status values that can be queried using qdstat(8) or qdmanage(8) tools. +
+

3.4. Configuration Entities

Configuration entities define the attributes allowed in the configuration file +(see qdrouterd.conf(5)) but you can also create entities once the router is +running using the qdrouterd(8) tool’s create operation. Some entities can also +be modified using the update operation, see the entity descriptions below.

3.4.1. container

(DEPRECATED)Attributes related to the AMQP container. This entity has been deprecated. Use the router entity instead.

Operations allowed: READ

+containerName (string, CREATE) +: +
+ The name of the AMQP container. If not specified, the container name will be set to a value of the container’s choosing. The automatically assigned container name is not guaranteed to be persistent across restarts of the container. +
+
+workerThreads (integer, default=4, CREATE) +: +
+ The number of threads that will be created to process message traffic and other application work (timers, non-amqp file descriptors, etc.) . +
+
+debugDump (path, CREATE) +: +
+ A file to dump debugging information that can’t be logged normally. +
+
+saslConfigPath (path, CREATE) +: +
+ Absolute path to the SASL configuration file. +
+
+saslConfigName (string, CREATE) +: +
+ Name of the SASL configuration. This string + .conf is the name of the configuration file. +
+

3.4.2. router

Tracks peer routers and computes routes to destinations.

Operations allowed: READ

+routerId (string, CREATE) +: +
+ (DEPRECATED) Router’s unique identity. This attribute has been deprecated. Use id instead +
+
+id (string, CREATE) +: +
+ Router’s unique identity. One of id or routerId is required. The router will fail to start without id or routerId +
+
+mode (One of [standalone, interior], default=standalone, CREATE) +: +
+ In standalone mode, the router operates as a single component. It does not participate in the routing protocol and therefore will not cooperate with other routers. In interior mode, the router operates in cooperation with other interior routers in an interconnected network. +
+
+area (string) +: +
+ Unused placeholder. +
+
+helloInterval (integer, default=1, CREATE) +: +
+ Interval in seconds between HELLO messages sent to neighbor routers. +
+
+helloMaxAge (integer, default=3, CREATE) +: +
+ Time in seconds after which a neighbor is declared lost if no HELLO is received. +
+
+raInterval (integer, default=30, CREATE) +: +
+ Interval in seconds between Router-Advertisements sent to all routers in a stable network. +
+
+raIntervalFlux (integer, default=4, CREATE) +: +
+ Interval in seconds between Router-Advertisements sent to all routers during topology fluctuations. +
+
+remoteLsMaxAge (integer, default=60, CREATE) +: +
+ Time in seconds after which link state is declared stale if no RA is received. +
+
+mobileAddrMaxAge (integer, default=60, CREATE) +: +
+ (DEPRECATED) This value is no longer used in the router. +
+
+addrCount (integer) +: +
+ Number of addresses known to the router. +
+
+linkCount (integer) +: +
+ Number of links attached to the router node. +
+
+nodeCount (integer) +: +
+ Number of known peer router nodes. +
+
+workerThreads (integer, default=4, CREATE) +: +
+ The number of threads that will be created to process message traffic and other application work (timers, non-amqp file descriptors, etc.) . +
+
+debugDump (path, CREATE) +: +
+ A file to dump debugging information that can’t be logged normally. +
+
+saslConfigPath (path, CREATE) +: +
+ Absolute path to the SASL configuration file. +
+
+saslConfigName (string, default=qdrouterd, CREATE) +: +
+ Name of the SASL configuration. This string + .conf is the name of the configuration file. +
+

3.4.3. listener

Listens for incoming connections to the router.

Operations allowed: CREATE, DELETE, READ

+addr (string, default=127.0.0.1, CREATE) +: +
+ (DEPRECATED)IP address: ipv4 or ipv6 literal or a host name. This attribute has been deprecated. Use host instead +
+
+host (string, default=127.0.0.1, CREATE) +: +
+ IP address: ipv4 or ipv6 literal or a host name +
+
+port (string, default=amqp, CREATE) +: +
+ Port number or symbolic service name. +
+
+protocolFamily (One of [IPv4, IPv6], CREATE) +: +
+ [IPv4, IPv6] IPv4: Internet Protocol version 4; IPv6: Internet Protocol version 6. If not specified, the protocol family will be automatically determined from the address. +
+
+role (One of [normal, inter-router, route-container, on-demand], default=normal, CREATE) +: +
+ The role of an established connection. In the normal role, the connection is assumed to be used for AMQP clients that are doing normal message delivery over the connection. In the inter-router role, the connection is assumed to be to another router in the network. Inter-router discovery and routing protocols can only be used over inter-router connections. route-container role can be used for router-container connections, for example, a router-broker connection. on-demand role has been deprecated. +
+
+cost (integer, default=1, CREATE) +: +
+ For the inter-router role only. This value assigns a cost metric to the inter-router connection. The default (and minimum) value is one. Higher values represent higher costs. The cost is used to influence the routing algorithm as it attempts to use the path with the lowest total cost from ingress to egress. +
+
+certDb (path, CREATE) +: +
+ The absolute path to the database that contains the public certificates of trusted certificate authorities (CA). +
+
+certFile (path, CREATE) +: +
+ The absolute path to the file containing the PEM-formatted public certificate to be used on the local end of any connections using this profile. +
+
+keyFile (path, CREATE) +: +
+ The absolute path to the file containing the PEM-formatted private key for the above certificate. +
+
+passwordFile (path, CREATE) +: +
+ If the above private key is password protected, this is the absolute path to a file containing the password that unlocks the certificate key. +
+
+password (string, CREATE) +: +
+ An alternative to storing the password in a file referenced by passwordFile is to supply the password right here in the configuration file. This option can be used by supplying the password in the password option. Don’t use both password and passwordFile in the same profile. +
+
+uidFormat (string, CREATE) +: +
+ A list of x509 client certificate fields that will be used to build a string that will uniquely identify the client certificate owner. For e.g. a value of cou indicates that the uid will consist of c - common name concatenated with o - organization-company name concatenated with u - organization unit; or a value of o2 indicates that the uid will consist of o (organization name) concatenated with 2 (the sha256 fingerprint of the entire certificate) . Allowed values can be any combination of http://git-wip-us.apache.org/repos/asf/qpid-site/blob/3e3e9ea1/content/releases/qpid-dispatch-master/book.pdf ---------------------------------------------------------------------- diff --git a/content/releases/qpid-dispatch-master/book.pdf b/content/releases/qpid-dispatch-master/book.pdf new file mode 100755 index 0000000..dbd7f6e Binary files /dev/null and b/content/releases/qpid-dispatch-master/book.pdf differ --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscribe@qpid.apache.org For additional commands, e-mail: commits-help@qpid.apache.org

Project

Messaging APIs

Servers and tools

Resources

1. Introduction

1.1. Overview

1.2. Benefits

1.3. Features

2. Using Qpid Dispatch

2.1. Configuration

2.2. Tools

2.2.1. qdstat

2.2.2. qdmanage

2.3. Basic Usage and Examples

2.3.1. Standalone and Interior Modes

2.3.2. Mobile Subscribers

2.3.3. Dynamic Reply-To

2.4. Link Routing

2.4.1. Configuration

2.5. Indirect Waypoints and Auto-Links

2.5.1. Queue Waypoint Example

2.5.2. Sharded Queue Example

2.5.3. Dynamically Adding Shards

3. Technical Details and Specifications

3.1. Client Compatibility

3.2. Addressing

3.2.1. Routing patterns

3.2.2. Routing mechanisms

Message routing

3.3. AMQP Mapping

3.3.1. Message Annotations

3.3.2. Source/Target Capabilities

3.3.3. Dynamic-Node-Properties

3.3.4. Addresses and Address Formats

Address Patterns

Supported Addresses

3.3.5. Implementation of the AMQP Management Specification

3.4. Configuration Entities

3.4.1. container

3.4.2. router

3.4.3. listener