qpid-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From c...@apache.org
Subject qpid-dispatch git commit: DISPATCH-549: Doc book needs more detailed introduction of system concepts
Date Thu, 27 Oct 2016 18:52:56 GMT
Repository: qpid-dispatch
Updated Branches:
  refs/heads/master 9a1f52f66 -> 4bd5e894b

DISPATCH-549: Doc book needs more detailed introduction of system concepts

Add a Theory of Operation chapter.

Project: http://git-wip-us.apache.org/repos/asf/qpid-dispatch/repo
Commit: http://git-wip-us.apache.org/repos/asf/qpid-dispatch/commit/4bd5e894
Tree: http://git-wip-us.apache.org/repos/asf/qpid-dispatch/tree/4bd5e894
Diff: http://git-wip-us.apache.org/repos/asf/qpid-dispatch/diff/4bd5e894

Branch: refs/heads/master
Commit: 4bd5e894b584ec05028557b5ae7c6b35a520d265
Parents: 9a1f52f
Author: Chuck Rolke <crolke@redhat.com>
Authored: Thu Oct 27 14:51:02 2016 -0400
Committer: Chuck Rolke <crolke@redhat.com>
Committed: Thu Oct 27 14:51:02 2016 -0400

 doc/book/book.adoc                |   1 +
 doc/book/theory_of_operation.adoc | 345 +++++++++++++++++++++++++++++++++
 2 files changed, 346 insertions(+)

diff --git a/doc/book/book.adoc b/doc/book/book.adoc
index e0618d8..eae0d17 100644
--- a/doc/book/book.adoc
+++ b/doc/book/book.adoc
@@ -24,6 +24,7 @@ Qpid Dispatch Router Book
 :doctype: book
 Using Qpid Dispatch

diff --git a/doc/book/theory_of_operation.adoc b/doc/book/theory_of_operation.adoc
new file mode 100644
index 0000000..487a084
--- /dev/null
+++ b/doc/book/theory_of_operation.adoc
@@ -0,0 +1,345 @@
+Licensed to the Apache Software Foundation (ASF) under one
+or more contributor license agreements.  See the NOTICE file
+distributed with this work for additional information
+regarding copyright ownership.  The ASF licenses this file
+to you under the Apache License, Version 2.0 (the
+"License"); you may not use this file except in compliance
+with the License.  You may obtain a copy of the License at
+  http://www.apache.org/licenses/LICENSE-2.0
+Unless required by applicable law or agreed to in writing,
+software distributed under the License is distributed on an
+KIND, either express or implied.  See the License for the
+specific language governing permissions and limitations
+under the License
+== Theory of Operation
+This section introduces some key concepts about the router.
+=== Overview
+The router is an _application layer_ program running as a normal
+user program or as a daemon.
+The router accepts AMQP connections from clients and creates AMQP
+connections to brokers or AMQP-based services.  The router classifies
+incoming AMQP messages and routes the
+messages between message producers and message consumers.
+The router is meant to be deployed in topologies of multiple routers,
+preferably with redundant paths. It uses link-state routing protocols
+and algorithms similar to OSPF or IS-IS from the networking world to
+calculate the best path from every message source to every message
+destination and to recover quickly from failures. The router relies on
+redundant network paths to provide continued connectivity in the face
+of system or network failure.
+A messaging client can make a single AMQP connection into a messaging
+bus built with routers and, over that connection, exchange messages
+with one or more message brokers connected to any router in the
+network. At the same time the client can exchange messages directly
+with other endpoints without involving a broker at all.
+=== Connections
+The router connects clients, servers, AMQP services, and other
+routers through network connections.
+==== Listener
+The router provides _listeners_ that accept client connections.
+A client connecting to a router listener uses the
+same methods that it would use to connect to a broker. From the
+client's perspective the router connection and link establishment are
+identical to broker connection and link establishment.
+Several types of listeners are defined by their role.
+| Role | Description
+| normal | The connection is used for AMQP clients using normal message delivery.
+| inter-router | The connection is assumed to be to another router in the network.  Inter-router
discovery and routing protocols can only be used over inter-router connections.
+| route-container | The connection is a broker or other resource that holds known addresses.
The router will use this connection to create links as necessary. The addresses are available
for routing only after the remote resource has created a connection.
+==== Connector
+The router can also be configured to create outbound connections to
+messaging brokers or other AMQP entities using _connectors_. A
+connector is defined with the network address of the broker and the
+name or names of the resources that are available in that broker. When
+a router connects to a broker through a connector it uses the same
+methods a normal messaging client would use when connecting to the
+Several types of connectors are defined by their role.
+| Role | Description
+| normal | The connection is used for AMQP clients using normal message delivery. On this
connector the router will initiate the connection but it will never create any links. Links
are to be created by the peer that accepts the connection.
+| inter-router | The connection is assumed to be to another router in the network.  Inter-router
discovery and routing protocols can only be used over inter-router connections.
+| route-container | The connection is to a broker or other resource that holds known addresses.
The router will use this connection to create links as necessary. The addresses are available
for routing only after the router has created a connection to the remote resource.
+=== Addresses
+AMQP addresses are used to control the flow of messages across a
+network of routers. Addresses are used in a number of different places
+in the AMQP 1.0 protocol. They can be used in a specific message in
+the _to_ and _reply-to_ fields of a message's properties. They are also
+used during the creation of links in the _address_ field of a _source_ or
+a _target_.
+Addresses in this discussion refer to AMQP protocol addresses and not
+to TCP/IP network addresses. TCP/IP network addresses are used by
+messaging clients, brokers, and routers to create AMQP connections.
+AMQP protocol addresses are the names of source and destination
+endpoints for messages within the messaging network.
+Addresses designate various kinds of entities in a messaging network:
+* Endpoint processes that consume data or offer a service
+* Topics that match multiple consumers to multiple producers
+* Entities within a messaging broker:
+** Queues
+** Durable Topics
+** Exchanges
+The syntax of an AMQP address is opaque as far as the router network
+is concerned. A syntactical structure may be used by the administrator
+who creates addresses but the router treats them as opaque
+The router maintains several classes of address based on how the address is
+configured or discovered.
+| Address Type | Description
+| mobile | The address is a rendezvous point between senders and receivers. The router aggregates
and serializes messages from senders and distributes messages to receivers.
+| link route | The address defines a private messaging path between a sender and a receiver.
The router simply passes messages between the end points.
+==== Mobile Addresses
+Routers consider addresses to be mobile such that any users of an
+address may be directly connected to any router in a network and may
+move around the topology. In cases where messages are broadcast to or
+balanced across multiple consumers, the address users may be connected
+to multiple routers in the network.
+Mobile addresses are rendezvous points for senders and receivers.
+Messages arrive at the mobile address and are dispatched to their
+destinations according to the routing defined for the mobile address.
+The details of these routing patterns are discussed later.
+Mobile addresses may be discovered during normal router operation or
+configured through management settings.
+===== Discovered Mobile Addresses
+Mobile addresses are created when a client creates a link to a source
+or destination address that is unknown to the router network.
+Suppose a service provider wants to offer _my-service_ that clients
+may use. The service provider must open a receiver link with source
+address _my-service_.  The router creates a mobile address
+_my-service_ and propagates the address so that it is known to every
+router in the network.
+Later a client wants to use the service and creates a sending link
+with target address _my-service_. The router matches the service
+provider's receiver having source address _my-service_ to the client's
+sender having target address _my-service_ and routes messages between
+the two.
+Any number of other clients can create links to the service as
+well. The clients do not have to know where in the router network the
+service provider is physically located nor are the clients required to
+connect to a specific router to use the service. Regardless of how
+many clients are using the service the service provider needs only a
+single connection and link into the router network.
+Another view of this same scenario is when a client tries to use the
+service before service provider has connected to the network. In this
+case the router network creates the mobile address _my-service_ as
+before. However, since the mobile address has only client sender links
+and no receiver links the router stalls the clients and prevents them
+from sending any messages.  Later, after the service provider connects
+and creates the receiver link, the router will issue credits to the
+clients and the messages will begin to flow between the clients and
+the service.
+The service provider can connect, disconnect, and reconnect from a
+different location without having to change any of the clients or
+their connections.  Imagine having the service running on a
+laptop. One day the connection is from corporate headquarters and the
+next day the connection is from some remote location. In this case the
+service provider's computer will typically have different host IP
+addresses for each connection. Using the router network the service
+provider connects to the router network and offers the named service
+and the clients connect to the router network and consume from the
+named service. The router network routes messages between the mobile
+addresses effectively masking host IP addresses of the service
+provider and the client systems.
+===== Configured Mobile Addresses
+Mobile addresses may be configured using the router _autoLink_
+object. An address created via an _autoLink_ represents a queue,
+topic, or other service in an external broker. Logically the
+_autoLink_ addresses are treated by the router network as if the
+broker had connected to the router and offered the services itself.
+For each configured mobile address the router will create a single
+link to the external resource. Messages flow between sender links and
+receiver links the same regardless if the mobile address was
+discovered or configured.
+Multiple _autoLink_ objects may define the same address on multiple
+brokers.  In this case the router network creates a sharded resource
+split between the brokers. Any client can seamlessly send and receive
+messages from either broker.
+Note that the brokers do not need to be clustered or federated to
+receive this treatment. The brokers may even be from different vendors
+or be different versions of the same broker yet still work together to
+provide a larger service platform.
+==== Link Route Addresses
+Link route addresses may be configured using the router _linkRoute_
+object. An link route address represents a queue, topic, or other
+service in an external broker similar to addresses configured by
+_autoLink_ objects. For link route addresses the router propagates a
+separate link attachment to the broker resource for each incoming
+client link. The router does not automatically create any links to the
+broker resource.
+Using link route addresses the router network does not participate in
+aggregated message distribution. The router simply passes message
+delivery and settlement between the two end points.
+=== Message Routing
+Addresses have semantics associated with them that are assigned when
+the address is provisioned or discovered.  The semantics of an address
+control how routers behave when they see the address being
+used. Address semantics include the following considerations:
+* Routing pattern - balanced, closest, multicast
+* Routing mechanism - message routed, link routed
+// * TODO: describe these???
+// * Undeliverable action - drop, hold and retry, redirect
+// * Reliability - N destinations, etc.
+==== Routing Patterns
+Routing patterns define the paths that a message with a mobile address can take across a
network. These routing patterns can be used for both direct routing, in which the router distributes
messages between clients without a broker, and indirect routing, in which the router enables
clients to exchange messages through a broker.
+|Pattern | Description
+| Balanced  | An anycast method which allows multiple receivers to use the same address.
In this case, messages (or links) are routed to exactly one of the receivers and the network
attempts to balance the traffic load across the set of receivers using the same address. This
routing delivers messages to receivers based on how quickly they settle the deliveries. Faster
receivers get more messages.
+| Closest | An anycast method in which even if there are more receivers for the same address,
every message is sent along the shortest path to reach the destination. This means that only
one receiver will get the message. Each message is delivered to the closest receivers in terms
of topology cost. If there are multiple receivers with the same lowest cost, deliveries will
be spread evenly among those receivers.
+| Multicast | Having multiple consumers on the same address at the same time, messages are
routed such that each consumer receives one copy of the message.
+==== Routing Mechanisms
+The fact that addresses can be used in different ways suggests that
+message routing can be accomplished in different ways. Before going
+into the specifics of the different routing mechanisms, it would be
+good to first define what is meant by the term _routing_:
+    In a network built of multiple, interconnected routers 'routing'
+    determines which connection to use to send a message directly
+    to its destination or one step closer to its destination.
+Each router serves as the terminus of a collection of incoming and
+outgoing links. Some of the links are designated for message routing,
+and others are designated for link routing. In both cases, the links
+either connect directly to endpoints that produce and consume
+messages, or they connect to other routers in the network along
+previously established connections.
+===== Message Routed
+Message routing occurs upon delivery of a message and is done based on
+the address in the message's _to_ field.
+When a delivery arrives on an incoming message-routing link, the
+router extracts the address from the delivered message's _to_ field and
+looks the address up in its routing table. The lookup results in zero
+or more outgoing links onto which the message shall be resent.
+Message routing can also occur without an address in the
+message's _to_ field if the incoming link has a target address.  In
+fact, if the sender uses a link with a target address, the _to_ field
+shall be ignored even if used.
+===== Link Routed
+Link routing occurs when a new link is attached to the router across
+one of its AMQP connections. It is done based on the _target.address_
+field of an inbound link and the _source.address_ field of an outbound
+Link routing uses the same routing table that message routing
+uses. The difference is that the routing occurs during the link-attach
+operation, and link attaches are propagated along the appropriate path
+to the destination. What results is a chain of links, connected
+end-to-end, from source to destination. It is similar to a virtual
+circuit in a telecom system.
+Each router in the chain holds pairs of link termini that are tied
+together. The router then simply exchanges all deliveries, delivery
+state changes, and link state changes between the two termini.
+The endpoints that use the link chain do not see any difference in
+behavior between a link chain and a single point-to-point link. All of
+the features available in the link protocol (flow control,
+transactional delivery, etc.) are available over a routed link-chain.
+==== Message Settlement
+Messages may be delivered with varying degrees of reliability.
+* At most once
+* At least once
+* Exactly once
+The reliability is negotiated between the client and server during
+link establishment. The router handles all levels of reliability by treating
+messages as either _pre-settled_ or _unsettled_.
+| Delivery | Handling
+| pre-settled | If the arriving delivery is pre-settled (i.e., fire and forget), the incoming
delivery shall be settled by the router, and the outgoing deliveries shall also be pre-settled.
In other words, the pre-settled nature of the message delivery is propagated across the network
to the message's destination.
+| unsettled | Unsettled delivery is also propagated across the network. Because unsettled
delivery records cannot be discarded, the router tracks the incoming deliveries and keeps
the association of the incoming deliveries to the resulting outgoing deliveries. This kept
association allows the router to continue to propagate changes in delivery state (settlement
and disposition) back and forth along the path which the message traveled.
+=== Security
+The router uses the SSL protocol and related certificates and SASL
+protocol mechanisms to encrypt and authenticate remote peers. Router
+listeners act as network servers and router connectors act as network
+clients. Both connection types may be configured securely with SSL
+and SASL.
+The router Policy module is an optional authorization mechanism
+enforcing user connection restrictions and AMQP resource access

To unsubscribe, e-mail: commits-unsubscribe@qpid.apache.org
For additional commands, e-mail: commits-help@qpid.apache.org

View raw message