ch02s03.html - Apache Qpid™

+ + +

+ + + + + +

+ + +

2.3. + QMF Python Console Tutorial +
Prev	Chapter 2. + Managing the AMQP Messaging Broker +

2.3. + QMF Python Console Tutorial +

2.3.1. + Prerequisite + - Install Qpid Messaging +

+ QMF uses AMQP Messaging (QPid) as its means of communication. To + use QMF, Qpid messaging must be installed somewhere in the + network. Qpid can be downloaded as source from Apache, is + packaged with a number of Linux distributions, and can be + purchased from commercial vendors that use Qpid. Please see + http://qpid.apache.orgfor + information as to where to get Qpid Messaging. +

+ Qpid Messaging includes a message broker (qpidd) which typically + runs as a daemon on a system. It also includes client bindings in + various programming languages. The Python-language client library + includes the QMF console libraries needed for this tutorial. +

+ Please note that Qpid Messaging has two broker implementations. + One is implemented in C++ and the other in Java. At press time, + QMF is supported only by the C++ broker. +

+ If the goal is to get the tutorial examples up and running as + quickly as possible, all of the Qpid components can be installed + on a single system (even a laptop). For more realistic + deployments, the broker can be deployed on a server and the + client/QMF libraries installed on other systems. +

2.3.2. + Synchronous + Console Operations +

+ The Python console API for QMF can be used in a synchronous + style, an asynchronous style, or a combination of both. + Synchronous operations are conceptually simple and are well + suited for user-interactive tasks. All operations are performed + in the context of a Python function call. If communication over + the message bus is required to complete an operation, the + function call blocks and waits for the expected result (or + timeout failure) before returning control to the caller. +

2.3.2.1. + Creating a QMF Console Session and Attaching to a Broker +

+ For the purposes of this tutorial, code examples will be shown as + they are entered in an interactive python session. +

+$ python
+Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38) 
+[GCC 4.3.2 20080917 (Red Hat 4.3.2-4)] on linux2
+Type "help", "copyright", "credits" or "license" for more information.
+>>> 
+

+ We will begin by importing the required libraries. If the Python + client is properly installed, these libraries will be found + normally by the Python interpreter. +

+>>> from qmf.console import Session
+

+ We must now create a Session object to manage this QMF + console session. +

+>>> sess = Session()
+

+ If no arguments are supplied to the creation of Session, + it defaults to synchronous-only operation. It also defaults to + user-management of connections. More on this in a moment. +

+ We will now establish a connection to the messaging broker. If + the broker daemon is running on the local host, simply use the + following: +

+>>> broker = sess.addBroker()
+

+ If the messaging broker is on a remote host, supply the URL to + the broker in the addBroker function call. Here's how to + connect to a local broker using the URL. +

+>>> broker = sess.addBroker("amqp://localhost")
+

+ The call to addBroker is synchronous and will return + only after the connection has been successfully established or + has failed. If a failure occurs, addBroker will raise an + exception that can be handled by the console script. +

+>>> try:
+...   broker = sess.addBroker("amqp://localhost:1000")
+... except:
+...   print "Connection Failed"
+... 
+Connection Failed
+>>> 
+

+ This operation fails because there is no Qpid Messaging broker + listening on port 1000 (the default port for qpidd is 5672). +

+ If preferred, the QMF session can manage the connection for you. + In this case, addBroker returns immediately and the + session attempts to establish the connection in the background. + This will be covered in detail in the section on asynchronous + operations. +

2.3.2.2. + Accessing + Managed Objects +

+ The Python console API provides access to remotely managed + objects via a proxy model. The API gives the client an + object that serves as a proxy representing the "real" object + being managed on the agent application. Operations performed on + the proxy result in the same operations on the real object. +

+ The following examples assume prior knowledge of the kinds of + objects that are actually available to be managed. There is a + section later in this tutorial that describes how to discover + what is manageable on the QMF bus. +

+ Proxy objects are obtained by calling the + Session.getObjects function. +

+ To illustrate, we'll get a list of objects representing queues in + the message broker itself. +

+>>> queues = sess.getObjects(_class="queue", _package="org.apache.qpid.broker")
+

+ queues is an array of proxy objects representing real + queues on the message broker. A proxy object can be printed to + display a description of the object. +

+>>> for q in queues:
+...   print q
+... 
+org.apache.qpid.broker:queue[0-1537-1-0-58] 0-0-1-0-1152921504606846979:reply-localhost.localdomain.32004
+org.apache.qpid.broker:queue[0-1537-1-0-61] 0-0-1-0-1152921504606846979:topic-localhost.localdomain.32004
+>>> 
+

+ Viewing Properties and Statistics of an Object +

+ Let us now focus our attention on one of the queue objects. +

+>>> queue = queues[0]
+

+ The attributes of an object are partitioned into + properties and statistics. Though the + distinction is somewhat arbitrary, properties tend to + be fairly static and may also be large and statistics + tend to change rapidly and are relatively small (counters, etc.). +

+ There are two ways to view the properties of an object. An array + of properties can be obtained using the getProperties + function: +

+>>> props = queue.getProperties()
+>>> for prop in props:
+...   print prop
+... 
+(vhostRef, 0-0-1-0-1152921504606846979)
+(name, u'reply-localhost.localdomain.32004')
+(durable, False)
+(autoDelete, True)
+(exclusive, True)
+(arguments, {})
+>>> 
+

+ The getProperties function returns an array of tuples. + Each tuple consists of the property descriptor and the property + value. +

+ A more convenient way to access properties is by using the + attribute of the proxy object directly: +

+>>> queue.autoDelete
+True
+>>> queue.name
+u'reply-localhost.localdomain.32004'
+>>> 
+

+ Statistics are accessed in the same way: +

+>>> stats = queue.getStatistics()
+>>> for stat in stats:
+...   print stat
+... 
+(msgTotalEnqueues, 53)
+(msgTotalDequeues, 53)
+(msgTxnEnqueues, 0)
+(msgTxnDequeues, 0)
+(msgPersistEnqueues, 0)
+(msgPersistDequeues, 0)
+(msgDepth, 0)
+(byteDepth, 0)
+(byteTotalEnqueues, 19116)
+(byteTotalDequeues, 19116)
+(byteTxnEnqueues, 0)
+(byteTxnDequeues, 0)
+(bytePersistEnqueues, 0)
+(bytePersistDequeues, 0)
+(consumerCount, 1)
+(consumerCountHigh, 1)
+(consumerCountLow, 1)
+(bindingCount, 2)
+(bindingCountHigh, 2)
+(bindingCountLow, 2)
+(unackedMessages, 0)
+(unackedMessagesHigh, 0)
+(unackedMessagesLow, 0)
+(messageLatencySamples, 0)
+(messageLatencyMin, 0)
+(messageLatencyMax, 0)
+(messageLatencyAverage, 0)
+>>> 
+

+ or alternatively: +

+>>> queue.byteTotalEnqueues
+19116
+>>>
+

+ The proxy objects do not automatically track changes that occur + on the real objects. For example, if the real queue enqueues more + bytes, viewing the byteTotalEnqueues statistic will show + the same number as it did the first time. To get updated data on + a proxy object, use the update function call: +

+>>> queue.update()
+>>> queue.byteTotalEnqueues
+19783
+>>>
+

Be Advised

+ The update method was added after the M4 release + of Qpid/Qmf. It may not be available in your + distribution. +

+ Invoking + Methods on an Object +

+ Up to this point, we have used the QMF Console API to find + managed objects and view their attributes, a read-only activity. + The next topic to illustrate is how to invoke a method on a + managed object. Methods allow consoles to control the managed + agents by either triggering a one-time action or by changing the + values of attributes in an object. +

+ First, we'll cover some background information about methods. A + QMF object class (of which a QMF object is an + instance), may have zero or more methods. To obtain a list of + methods available for an object, use the getMethods + function. +

+>>> methodList = queue.getMethods()
+

+ getMethods returns an array of method descriptors (of + type qmf.console.SchemaMethod). To get a summary of a method, you + can simply print it. The _repr_ function returns a + string that looks like a function prototype. +

+>>> print methodList
+[purge(request)]
+>>>
+

+ For the purposes of illustration, we'll use a more interesting + method available on the broker object which represents + the connected Qpid message broker. +

+>>> br = sess.getObjects(_class="broker", _package="org.apache.qpid.broker")[0]
+>>> mlist = br.getMethods()
+>>> for m in mlist:
+...   print m
+... 
+echo(sequence, body)
+connect(host, port, durable, authMechanism, username, password, transport)
+queueMoveMessages(srcQueue, destQueue, qty)
+>>>
+

+ We have just learned that the broker object has three + methods: echo, connect, and + queueMoveMessages. We'll use the echo method to + "ping" the broker. +

+>>> result = br.echo(1, "Message Body")
+>>> print result
+OK (0) - {'body': u'Message Body', 'sequence': 1}
+>>> print result.status
+0
+>>> print result.text
+OK
+>>> print result.outArgs
+{'body': u'Message Body', 'sequence': 1}
+>>>
+

+ In the above example, we have invoked the echo method on + the instance of the broker designated by the proxy "br" with a + sequence argument of 1 and a body argument of "Message Body". The + result indicates success and contains the output arguments (in + this case copies of the input arguments). +

+ To be more precise... Calling echo on the proxy causes + the input arguments to be marshalled and sent to the remote agent + where the method is executed. Once the method execution + completes, the output arguments are marshalled and sent back to + the console to be stored in the method result. +

+ You are probably wondering how you are supposed to know what + types the arguments are and which arguments are input, which are + output, or which are both. This will be addressed later in the + "Discovering what Kinds of Objects are Available" section. +

2.3.3. + Asynchronous + Console Operations +

+ QMF is built on top of a middleware messaging layer (Qpid + Messaging). Because of this, QMF can use some communication + patterns that are difficult to implement using network transports + like UDP, TCP, or SSL. One of these patterns is called the + Publication and Subscription pattern (pub-sub for + short). In the pub-sub pattern, data sources publish + information without a particular destination in mind. Data sinks + (destinations) subscribe using a set of criteria that + describes what kind of data they are interested in receiving. + Data published by a source may be received by zero, one, or many + subscribers. +

+ QMF uses the pub-sub pattern to distribute events, object + creation and deletion, and changes to properties and statistics. + A console application using the QMF Console API can receive these + asynchronous and unsolicited events and updates. This is useful + for applications that store and analyze events and/or statistics. + It is also useful for applications that react to certain events + or conditions. +

+ Note that console applications may always use the synchronous + mechanisms. +

2.3.3.1. + Creating a Console Class to Receive Asynchronous Data +

+ Asynchronous API operation occurs when the console application + supplies a Console object to the session manager. The + Console object (which overrides the + qmf.console.Console class) handles all asynchronously + arriving data. The Console class has the following + methods. Any number of these methods may be overridden by the + console application. Any method that is not overridden defaults + to a null handler which takes no action when invoked. +

Table 2.4. QMF Python Console Class Methods

+ Method +	+ Arguments +	+ Invoked when... +
+ brokerConnected +	+ broker +	+ a connection to a broker is established +
+ brokerDisconnected +	+ broker +	+ a connection to a broker is lost +
+ newPackage +	+ name +	+ a new package is seen on the QMF bus +
+ newClass +	+ kind, classKey +	+ a new class (event or object) is seen on the QMF bus +
+ newAgent +	+ agent +	+ a new agent appears on the QMF bus +
+ delAgent +	+ agent +	+ an agent disconnects from the QMF bus +
+ objectProps +	+ broker, object +	+ the properties of an object are published +
+ objectStats +	+ broker, object +	+ the statistics of an object are published +
+ event +	+ broker, event +	+ an event is published +
+ heartbeat +	+ agent, timestamp +	+ a heartbeat is published by an agent +
+ brokerInfo +	+ broker +	+ information about a connected broker is available to be + queried +
+ methodResponse +	+ broker, seq, response +	+ the result of an asynchronous method call is received +

+ Supplied with the API is a class called DebugConsole. + This is a test Console instance that overrides all of + the methods such that arriving asynchronous data is printed to + the screen. This can be used to see all of the arriving + asynchronous data. +

2.3.3.2. + Receiving + Events +

+ We'll start the example from the beginning to illustrate the + reception and handling of events. In this example, we will create + a Console class that handles broker-connect, + broker-disconnect, and event messages. We will also allow the + session manager to manage the broker connection for us. +

+ Begin by importing the necessary classes: +

+>>> from qmf.console import Session, Console
+

+ Now, create a subclass of Console that handles the three + message types: +

+>>> class EventConsole(Console):
+...   def brokerConnected(self, broker):
+...     print "brokerConnected:", broker
+...   def brokerDisconnected(self, broker):
+...     print "brokerDisconnected:", broker
+...   def event(self, broker, event):
+...     print "event:", event
+...
+>>>
+

+ Make an instance of the new class: +

+>>> myConsole = EventConsole()
+

+ Create a Session class using the console instance. In + addition, we shall request that the session manager do the + connection management for us. Notice also that we are requesting + that the session manager not receive objects or heartbeats. Since + this example is concerned only with events, we can optimize the + use of the messaging bus by telling the session manager not to + subscribe for object updates or heartbeats. +

+>>> sess = Session(myConsole, manageConnections=True, rcvObjects=False, rcvHeartbeats=False)
+>>> broker = sess.addBroker()
+>>>
+

+ Once the broker is added, we will begin to receive asynchronous + events (assuming there is a functioning broker available to + connect to). +

+brokerConnected: Broker connected at: localhost:5672
+event: Thu Jan 29 19:53:19 2009 INFO  org.apache.qpid.broker:bind broker=localhost:5672 ...
+

2.3.3.3. + Receiving + Objects +

+ To illustrate asynchronous handling of objects, a small console + program is supplied. The entire program is shown below for + convenience. We will then go through it part-by-part to explain + its design. +

+ This console program receives object updates and displays a set + of statistics as they change. It focuses on broker queue objects. +

+# Import needed classes
+from qmf.console import Session, Console
+from time        import sleep
+
+# Declare a dictionary to map object-ids to queue names
+queueMap = {}
+
+# Customize the Console class to receive object updates.
+class MyConsole(Console):
+
+  # Handle property updates
+  def objectProps(self, broker, record):
+
+    # Verify that we have received a queue object.  Exit otherwise.
+    classKey = record.getClassKey()
+    if classKey.getClassName() != "queue":
+      return
+
+    # If this object has not been seen before, create a new mapping from objectID to name
+    oid = record.getObjectId()
+    if oid not in queueMap:
+      queueMap[oid] = record.name
+
+  # Handle statistic updates
+  def objectStats(self, broker, record):
+    
+    # Ignore updates for objects that are not in the map
+    oid = record.getObjectId()
+    if oid not in queueMap:
+      return
+
+    # Print the queue name and some statistics
+    print "%s: enqueues=%d dequeues=%d" % (queueMap[oid], record.msgTotalEnqueues, record.msgTotalDequeues)
+
+    # if the delete-time is non-zero, this object has been deleted.  Remove it from the map.
+    if record.getTimestamps()[2] > 0:
+      queueMap.pop(oid)
+
+# Create an instance of the QMF session manager.  Set userBindings to True to allow
+# this program to choose which objects classes it is interested in.
+sess = Session(MyConsole(), manageConnections=True, rcvEvents=False, userBindings=True)
+
+# Register to receive updates for broker:queue objects.
+sess.bindClass("org.apache.qpid.broker", "queue")
+broker = sess.addBroker()
+
+# Suspend processing while the asynchronous operations proceed.
+try:
+  while True:
+    sleep(1)
+except:
+  pass
+
+# Disconnect the broker before exiting.
+sess.delBroker(broker)
+

+ Before going through the code in detail, it is important to + understand the differences between synchronous object access and + asynchronous object access. When objects are obtained + synchronously (using the getObjects function), the + resulting proxy contains all of the object's attributes, both + properties and statistics. When object data is published + asynchronously, the properties and statistics are sent separately + and only when the session first connects or when the content + changes. +

+ The script wishes to print the queue name with the updated + statistics, but the queue name is only present with the + properties. For this reason, the program needs to keep some state + to correlate property updates with their corresponding statistic + updates. This can be done using the ObjectId that + uniquely identifies the object. +

+    # If this object has not been seen before, create a new mapping from objectID to name
+    oid = record.getObjectId()
+    if oid not in queueMap:
+      queueMap[oid] = record.name
+

+ The above code fragment gets the object ID from the proxy and + checks to see if it is in the map (i.e. has been seen before). If + it is not in the map, a new map entry is inserted mapping the + object ID to the queue's name. +

+    # if the delete-time is non-zero, this object has been deleted.  Remove it from the map.
+    if record.getTimestamps()[2] > 0:
+      queueMap.pop(oid)
+

+ This code fragment detects the deletion of a managed object. + After reporting the statistics, it checks the timestamps of the + proxy. getTimestamps returns a list of timestamps in the + order: +

+ Current - The timestamp of the sending of this update. +
+ Create - The time of the object's creation +
+ Delete - The time of the object's deletion (or zero if + not deleted) +

+ This code structure is useful for getting information about + very-short-lived objects. It is possible that an object will be + created, used, and deleted within an update interval. In this + case, the property update will arrive first, followed by the + statistic update. Both will indicate that the object has been + deleted but a full accounting of the object's existence and final + state is reported. +

+# Create an instance of the QMF session manager.  Set userBindings to True to allow
+# this program to choose which objects classes it is interested in.
+sess = Session(MyConsole(), manageConnections=True, rcvEvents=False, userBindings=True)
+
+# Register to receive updates for broker:queue objects.
+sess.bindClass("org.apache.qpid.broker", "queue")
+

+ The above code is illustrative of the way a console application + can tune its use of the QMF bus. Note that rcvEvents is + set to False. This prevents the reception of events. Note also + the use of userBindings=True and the call to + sess.bindClass. If userBindings is set to False + (its default), the session will receive object updates for all + classes of object. In the case above, the application is only + interested in broker:queue objects and reduces its bus bandwidth + usage by requesting updates to only that class. + bindClass may be called as many times as desired to add + classes to the list of subscribed classes. +

2.3.3.4. + Asynchronous Method Calls and Method Timeouts +

+ Method calls can also be invoked asynchronously. This is useful + if a large number of calls needs to be made in a short time + because the console application will not need to wait for the + complete round-trip delay for each call. +

+ Method calls are synchronous by default. They can be made + asynchronous by adding the keyword-argument _async=True + to the method call. +

+ In a synchronous method call, the return value is the method + result. When a method is called asynchronously, the return value + is a sequence number that can be used to correlate the eventual + result to the request. This sequence number is passed as an + argument to the methodResponse function in the + Console interface. +

+ It is important to realize that the methodResponse + function may be invoked before the asynchronous call returns. + Make sure your code is written to handle this possibility. +

2.3.4. + Discovering what Kinds of Objects are Available +

Prev	Up
2.2. + Qpid Management Framework +	Home

+ +

+ + + +

+ Apache Qpid, Messaging built on AMQP; Copyright © 2015 + The Apache Software Foundation; Licensed under + the Apache + License, Version 2.0; Apache Qpid, Qpid, Qpid Proton, + Proton, Apache, the Apache feather logo, and the Apache Qpid + project logo are trademarks of The Apache Software + Foundation; All other marks mentioned may be trademarks or + registered trademarks of their respective owners +

+ + +

+ + + + + +

+ + +

1.4. Broker Federation
Prev	Chapter 1. + Running the AMQP Messaging Broker +	Next

1.4. Broker Federation

+ Broker Federation allows messaging networks to be defined by creating message routes, in which messages in one broker (the source broker) are automatically routed to another broker (the destination broker). These routes may be defined between exchanges in the two brokers (the source exchange and the destination exchange), or from a queue in the source broker (the source queue) to an exchange in the destination broker. Message routes are unidirectional; when bidirectional flow is needed, one route is created in each direction. Routes can be durable or transient. A durable route survives broker restarts, restoring a route as soon as both the source broker and the destination are available. If the connection to a destination is lost, messages associated with a durable route continue to accumulate on the source, so they can be retrieved when the connection is reestablished. +

+ Broker Federation can be used to build large messaging networks, with many brokers, one route at a time. If network connectivity permits, an entire distributed messaging network can be configured from a single location. The rules used for routing can be changed dynamically as servers change, responsibilities change, at different times of day, or to reflect other changing conditions. +

+ Broker Federation is useful in a wide variety of scenarios. Some of these have to do with functional organization; for instance, brokers may be organized by geography, service type, or priority. Here are some use cases for federation: +

+ Geography: Customer requests may be routed to a processing location close to the customer. +
+ Service Type: High value customers may be routed to more responsive servers. +
+ Load balancing: Routing among brokers may be changed dynamically to account for changes in actual or anticipated load. +
+ High Availability: Routing may be changed to a new broker if an existing broker becomes unavailable. +
+ WAN Connectivity: Federated routes may connect disparate locations across a wide area network, while clients connect to brokers on their own local area network. Each broker can provide persistent queues that can hold messages even if there are gaps in WAN connectivity. +
+ Functional Organization: The flow of messages among software subsystems can be configured to mirror the logical structure of a distributed application. +
+ Replicated Exchanges: High-function exchanges like the XML exchange can be replicated to scale performance. +
+ Interdepartmental Workflow: The flow of messages among brokers can be configured to mirror interdepartmental workflow at an organization. +

+ +

1.4.1. Message Routes

+ Broker Federation is done by creating message routes. The destination for a route is always an exchange on the destination broker. By default, a message route is created by configuring the destination broker, which then contacts the source broker to subscribe to the source queue. This is called a pull route. It is also possible to create a route by configuring the source broker, which then contacts the destination broker in order to send messages. This is called a push route, and is particularly useful when the destination broker may not be available at the time the messaging route is configured, or when a large number of routes are created with the same destination exchange. +

+ The source for a route can be either an exchange or a queue on the source broker. If a route is between two exchanges, the routing criteria can be given explicitly, or the bindings of the destination exchange can be used to determine the routing criteria. To support this functionality, there are three kinds of message routes: queue routes, exchange routes, and dynamic exchange routes. +

1.4.1.1. Queue Routes

+ Queue Routes route all messages from a source queue to a destination exchange. If message acknowledgement is enabled, messages are removed from the queue when they have been received by the destination exchange; if message acknowledgement is off, messages are removed from the queue when sent. +

1.4.1.2. Exchange Routes

+ Exchange routes route messages from a source exchange to a destination exchange, using a binding key (which is optional for a fanout exchange). +

+ Internally, creating an exchange route creates a private queue (auto-delete, exclusive) on the source broker to hold messages that are to be routed to the destination broker, binds this private queue to the source broker exchange, and subscribes the destination broker to the queue. +

1.4.1.3. Dynamic Exchange Routes

+ Dynamic exchange routes allow a client to create bindings to an exchange on one broker, and receive messages that satisfy the conditions of these bindings not only from the exchange to which the client created the binding, but also from other exchanges that are connected to it using dynamic exchange routes. If the client modifies the bindings for a given exchange, they are also modified for dynamic exchange routes associated with that exchange. +

+ Dynamic exchange routes apply all the bindings of a destination exchange to a source exchange, so that any message that would match one of these bindings is routed to the destination exchange. If bindings are added or removed from the destination exchange, these changes are reflected in the dynamic exchange route -- when the destination broker creates a binding with a given binding key, this is reflected in the route, and when the destination broker drops a binding with a binding key, the route no longer incurs the overhead of transferring messages that match the binding key among brokers. If two exchanges have dynamic exchange routes to each other, then all bindings in each exchange are reflected in the dynamic exchange route of the other. In a dynamic exchange route, the source and destination exchanges must have the same exchange type, and they must have the same name; for instance, if the source exchange is a direct exchange, the destination exchange must also be a direct exchange, and the names must match. +

+ Internally, dynamic exchange routes are implemented in the same way as exchange routes, except that the bindings used to implement dynamic exchange routes are modified if the bindings in the destination exchange change. +

+ A dynamic exchange route is always a pull route. It can never be a push route. +

1.4.2. Federation Topologies

+ A federated network is generally a tree, star, or line, using bidirectional links (implemented as a pair of unidirectional links) between any two brokers. A ring topology is also possible, if only unidirectional links are used. +

+ Every message transfer takes time. For better performance, you should minimize the number of brokers between the message origin and final destination. In most cases, tree or star topologies do this best. +

+ For any pair of nodes A,B in a federated network, there should be only one path from A to B. If there is more than one path, message loops can cause duplicate message transmission and flood the federated network. The topologies discussed above do not have message loops. A ring topology with bidirectional links is one example of a topology that does cause this problem, because a given broker can receive the same message from two different brokers. Mesh topologies can also cause this problem. +

1.4.3. Federation among High Availability Message Clusters

+ Federation is generally used together with High Availability Message Clusters, using clusters to provide high availability on each LAN, and federation to route messages among the clusters. Because message state is replicated within a cluster, it makes little sense to define message routes between brokers in the same cluster. +

+ To create a message route between two clusters, simply create a route between any one broker in the first cluster and any one broker in the second cluster. Each broker in a given cluster can use message routes defined for another broker in the same cluster. If the broker for which a message route is defined should fail, another broker in the same cluster can restore the message route. +

1.4.4. The qpid-route Utility

+ qpid-route is a command line utility used to configure federated networks of brokers and to view the status and topology of networks. It can be used to configure routes among any brokers that qpid-route can connect to. +

+ The syntax of qpid-route is as follows: +

+      qpid-route [OPTIONS] dynamic add <dest-broker> <src-broker> <exchange>
+      qpid-route [OPTIONS] dynamic del <dest-broker> <src-broker> <exchange>
+
+      qpid-route [OPTIONS] route add <dest-broker> <src-broker> <exchange> <routing-key>
+      qpid-route [OPTIONS] route del <dest-broker> <src-broker> <exchange> <routing-key>
+
+      qpid-route [OPTIONS] queue add <dest-broker> <src-broker> <dest-exchange>  <src-queue>
+      qpid-route [OPTIONS] queue del <dest-broker> <src-broker> <dest-exchange>  <src-queue>
+
+      qpid-route [OPTIONS] list  [<broker>]
+      qpid-route [OPTIONS] flush [<broker>]
+      qpid-route [OPTIONS] map   [<broker>]
+
+      
+      qpid-route [OPTIONS] list connections [<broker>]
+

+ The syntax for broker, dest-broker, and src-broker is as follows: +

+      [username/password@] hostname | ip-address [:<port>]
+

+ The following are all valid examples of the above syntax: localhost, 10.1.1.7:10000, broker-host:10000, guest/guest@localhost. +

+ These are the options for qpid-route: +

Table 1.9. qpid-route options

+ -v +	+ Verbose output. +
+ -q +	+ Quiet output, will not print duplicate warnings. +
+ -d +	+ Make the route durable. +
+ --timeout N +	+ Maximum time to wait when qpid-route connects to a broker, in seconds. Default is 10 seconds. +
+ --ack N +	+ Acknowledge transfers of routed messages in batches of N. Default is 0 (no acknowledgements). Setting to 1 or greater enables acknowledgements; when using acknowledgements, values of N greater than 1 can significnantly improve performance, especially if there is significant network latency between the two brokers. +
+ -s [ --src-local ] +	+ Configure the route in the source broker (create a push route). +
+ -t <transport> [ --transport <transport>] +	+ Transport protocol to be used for the route. + + tcp (default) + + ssl + + rdma + + +

1.4.4.1. Creating and Deleting Queue Routes

+ The syntax for creating and deleting queue routes is as follows: +

+	qpid-route [OPTIONS] queue add <dest-broker> <src-broker> <dest-exchange> <src-queue>
+	qpid-route [OPTIONS] queue del <dest-broker> <src-broker> <dest-exchange> <src-queue>
+

+ For instance, the following creates a queue route that routes all messages from the queue named public on the source broker localhost:10002 to the amq.fanout exchange on the destination broker localhost:10001: +

+	$ qpid-route queue add localhost:10001 localhost:10002 amq.fanout public
+

+ If the -d option is specified, this queue route is persistent, and will be restored if one or both of the brokers is restarted: +

+	$ qpid-route -d queue add localhost:10001 localhost:10002 amq.fanout public
+

+ The del command takes the same arguments as the add command. The following command deletes the queue route described above: +

+	$ qpid-route queue del localhost:10001 localhost:10002 amq.fanout public
+

1.4.4.2. Creating and Deleting Exchange Routes

+ The syntax for creating and deleting exchange routes is as follows: +

+	qpid-route [OPTIONS] route add <dest-broker> <src-broker> <exchange> <routing-key>
+	qpid-route [OPTIONS] route del <dest-broker> <src-broker> <exchange> <routing-key>
+	qpid-route [OPTIONS] flush [<broker>]
+

+ For instance, the following creates an exchange route that routes messages that match the binding key global.# from the amq.topic exchange on the source broker localhost:10002 to the amq.topic exchange on the destination broker localhost:10001: +

+	$ qpid-route route add localhost:10001 localhost:10002 amq.topic global.#
+

+ In many applications, messages published to the destination exchange should also be routed to the source exchange. This is accomplished by creating a second exchange route, reversing the roles of the two exchanges: +

+	$ qpid-route route add localhost:10002 localhost:10001 amq.topic global.#
+

+ If the -d option is specified, the exchange route is persistent, and will be restored if one or both of the brokers is restarted: +

+	$ qpid-route -d route add localhost:10001 localhost:10002 amq.fanout public
+

+ The del command takes the same arguments as the add command. The following command deletes the first exchange route described above: +

+	$ qpid-route route del localhost:10001 localhost:10002 amq.topic global.#
+

1.4.4.3. Deleting all routes for a broker

+ Use the flush command to delete all routes for a given broker: +

+	qpid-route [OPTIONS] flush [<broker>]
+

+ For instance, the following command deletes all routes for the broker localhost:10001: +

+	$ qpid-route flush localhost:10001
+

1.4.4.4. Creating and Deleting Dynamic Exchange Routes

+ The syntax for creating and deleting dynamic exchange routes is as follows: +

+	qpid-route [OPTIONS] dynamic add <dest-broker> <src-broker> <exchange>
+	qpid-route [OPTIONS] dynamic del <dest-broker> <src-broker> <exchange>
+

+ In the following examples, we will route messages from a topic exchange. We will create a new topic exchange and federate it so that we are not affected by other all clients that use the built-in amq.topic exchange. The following commands create a new topic exchange on each of two brokers: +

+	$ qpid-config -a localhost:10003 add exchange topic fed.topic
+	$ qpid-config -a localhost:10004 add exchange topic fed.topic
+

+ Now let's create a dynamic exchange route that routes messages from the fed.topic exchange on the source broker localhost:10004 to the fed.topic exchange on the destination broker localhost:10003 if they match any binding on the destination broker's fed.topic exchange: +

+	$ qpid-route dynamic add localhost:10003 localhost:10004 fed.topic
+

+ Internally, this creates a private autodelete queue on the source broker, and binds that queue to the fed.topic exchange on the source broker, using each binding associated with the fed.topic exchange on the destination broker. +

+ In many applications, messages published to the destination exchange should also be routed to the source exchange. This is accomplished by creating a second dynamic exchange route, reversing the roles of the two exchanges: +

+	$ qpid-route dynamic add localhost:10004 localhost:10003 fed.topic
+

+ If the -d option is specified, the exchange route is persistent, and will be restored if one or both of the brokers is restarted: +

+	$ qpid-route -d dynamic add localhost:10004 localhost:10003 fed.topic
+

+ When an exchange route is durable, the private queue used to store messages for the route on the source exchange is also durable. If the connection between the brokers is lost, messages for the destination exchange continue to accumulate until it can be restored. +

+ The del command takes the same arguments as the add command. The following command deletes the first exchange route described above: +

+	$ qpid-route dynamic del localhost:10004 localhost:10003 fed.topic
+

+ Internally, this deletes the bindings on the source exchange for the the private queues associated with the message route. +

1.4.4.5. Viewing Routes

+ The route list command shows the routes associated with an individual broker. For instance, suppose we have created the following two routes: +

+	$ qpid-route dynamic add localhost:10003 localhost:10004 fed.topic
+	$ qpid-route dynamic add localhost:10004 localhost:10003 fed.topic
+

+ We can now use route list to show all routes for the broker localhost:10003: +

+	$ qpid-route route list localhost:10003
+	localhost:10003 localhost:10004 fed.topic <dynamic>
+

+ Note that this shows only one of the two routes we created, the route for which localhost:10003 is a destination. If we want to see the route for which localhost:10004 is a destination, we need to do another route list: +

+	$ qpid-route route list localhost:10004
+	localhost:10004 localhost:10003 fed.topic <dynamic>
+

+ The route map command shows all routes associated with a broker, and recursively displays all routes for brokers involved in federation relationships with the given broker. For instance, here is the output for the two brokers configured above: +

+	$ qpid-route route map localhost:10003
+
+	Finding Linked Brokers:
+	localhost:10003... Ok
+	localhost:10004... Ok
+
+	Dynamic Routes:
+
+	Exchange fed.topic:
+	localhost:10004 <=> localhost:10003
+
+	Static Routes:
+	none found
+

+ Note that the two dynamic exchange links are displayed as though they were one bidirectional link. The route map command is particularly helpful for larger, more complex networks. Let's configure a somewhat more complex network with 16 dynamic exchange routes: +

+	qpid-route dynamic add localhost:10001 localhost:10002 fed.topic
+	qpid-route dynamic add localhost:10002 localhost:10001 fed.topic
+
+	qpid-route dynamic add localhost:10003 localhost:10002 fed.topic
+	qpid-route dynamic add localhost:10002 localhost:10003 fed.topic
+
+	qpid-route dynamic add localhost:10004 localhost:10002 fed.topic
+	qpid-route dynamic add localhost:10002 localhost:10004 fed.topic
+
+	qpid-route dynamic add localhost:10002 localhost:10005 fed.topic
+	qpid-route dynamic add localhost:10005 localhost:10002 fed.topic
+
+	qpid-route dynamic add localhost:10005 localhost:10006 fed.topic
+	qpid-route dynamic add localhost:10006 localhost:10005 fed.topic
+
+	qpid-route dynamic add localhost:10006 localhost:10007 fed.topic
+	qpid-route dynamic add localhost:10007 localhost:10006 fed.topic
+
+	qpid-route dynamic add localhost:10006 localhost:10008 fed.topic
+	qpid-route dynamic add localhost:10008 localhost:10006 fed.topic
+

+ Now we can use route map starting with any one broker, and see the entire network: +

+	$ ./qpid-route route map localhost:10001
+
+	Finding Linked Brokers:
+	localhost:10001... Ok
+	localhost:10002... Ok
+	localhost:10003... Ok
+	localhost:10004... Ok
+	localhost:10005... Ok
+	localhost:10006... Ok
+	localhost:10007... Ok
+	localhost:10008... Ok
+
+	Dynamic Routes:
+
+	Exchange fed.topic:
+	localhost:10002 <=> localhost:10001
+	localhost:10003 <=> localhost:10002
+	localhost:10004 <=> localhost:10002
+	localhost:10005 <=> localhost:10002
+	localhost:10006 <=> localhost:10005
+	localhost:10007 <=> localhost:10006
+	localhost:10008 <=> localhost:10006
+
+	Static Routes:
+	none found
+

1.4.4.6. Resilient Connections

+ When a broker route is created, or when a durable broker route is restored after broker restart, a connection is created between the source broker and the destination broker. The connections used between brokers are called resilient connections; if the connection fails due to a communication error, it attempts to reconnect. The retry interval begins at 2 seconds and, as more attempts are made, grows to 64 seconds, and continues to retry every 64 seconds thereafter. If the connection fails due to an authentication problem, it will not continue to retry. +

+ The command list connections can be used to show the resilient connections for a broker: +

+	$ qpid-route list connections localhost:10001
+
+	Host            Port    Transport Durable  State             Last Error
+	=============================================================================
+	localhost       10002   tcp          N     Operational
+	localhost       10003   tcp          N     Operational
+	localhost       10009   tcp          N     Waiting           Connection refused
+

+ In the above output, Last Error contains the string representation of the last connection error received for the connection. State represents the state of the connection, and may be one of the following values: +

Table 1.10. State values in $ qpid-route list connections

+ Waiting +	+ Waiting before attempting to reconnect. +
+ Connecting +	+ Attempting to establish the connection. +
+ Operational +	+ The connection has been established and can be used. +
+ Failed +	+ The connection failed and will not retry (usually because authentication failed). +
+ Closed +	+ The connection has been closed and will soon be deleted. +
+ Passive +	+ If a cluster is federated to another cluster, only one of the nodes has an actual connection to remote node. Other nodes in the cluster have a passive connection. +

1.4.5. Broker options affecting federation

+ The following broker options affect federation: +

Table 1.11. Broker Options for Federation

+ Options for Federation +
+ `federation-tag NAME` +	+ A unique name to identify this broker in federation network. + If not specified, the broker will generate a unique identifier. +
+ `link-maintenance-interval SECONDS` + ^[b] +	+ + Interval to check if links need to be re-connected. Default 2 + seconds. Can be a sub-second interval for faster failover, + e.g. 0.1 seconds. + +
+ `link-heartbeat-interval SECONDS` + ^[b] +	+ + Heart-beat interval for federation links. If no heart-beat is + received for twice the interval the link is considered dead. + Default 120 seconds. + +

Prev	Up	Next
1.3. + Cheat Sheet for configuring Exchange Options +	Home	1.5. Security

+ +

+ + + +

Project

Messaging APIs

Servers and tools

Resources

2.3. + QMF Python Console Tutorial +

2.3.1. + Prerequisite + - Install Qpid Messaging +

2.3.2. + Synchronous + Console Operations +

2.3.2.1. + Creating a QMF Console Session and Attaching to a Broker +

2.3.2.2. + Accessing + Managed Objects +

+ Viewing Properties and Statistics of an Object +

Be Advised

+ Invoking + Methods on an Object +

2.3.3. + Asynchronous + Console Operations +

2.3.3.1. + Creating a Console Class to Receive Asynchronous Data +

2.3.3.2. + Receiving + Events +

2.3.3.3. + Receiving + Objects +

2.3.3.4. + Asynchronous Method Calls and Method Timeouts +

2.3.4. + Discovering what Kinds of Objects are Available +

Project

Messaging APIs

Servers and tools

Resources

1.4. Broker Federation

1.4.1. Message Routes

1.4.1.1. Queue Routes

1.4.1.2. Exchange Routes

1.4.1.3. Dynamic Exchange Routes

1.4.2. Federation Topologies

1.4.3. Federation among High Availability Message Clusters

1.4.4. The qpid-route Utility

1.4.4.1. Creating and Deleting Queue Routes

1.4.4.2. Creating and Deleting Exchange Routes

1.4.4.3. Deleting all routes for a broker

1.4.4.4. Creating and Deleting Dynamic Exchange Routes

1.4.4.5. Viewing Routes

1.4.4.6. Resilient Connections

1.4.5. Broker options affecting federation