10.6. Client failover

Reply-To: dev@qpid.apache.org Delivered-To: mailing list commits@qpid.apache.org Received: (qmail 86631 invoked by uid 99); 5 Apr 2018 17:35:08 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 05 Apr 2018 17:35:08 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id EC8A7F6BDC; Thu, 5 Apr 2018 17:35:07 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: orudyy@apache.org To: commits@qpid.apache.org Date: Thu, 05 Apr 2018 17:35:51 -0000 Message-Id: <68eab128df234275bc3bad867eb15e56@git.apache.org> In-Reply-To: <8353861c88f940c085efbe8142b34cb4@git.apache.org> References: <8353861c88f940c085efbe8142b34cb4@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [45/51] [partial] qpid-site git commit: QPID-8149: Update site for Qpid Java release 6.1.6 http://git-wip-us.apache.org/repos/asf/qpid-site/blob/dc514b74/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-ClientFailover.html ---------------------------------------------------------------------- diff --git a/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-ClientFailover.html b/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-ClientFailover.html new file mode 100644 index 0000000..3462462 --- /dev/null +++ b/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-ClientFailover.html @@ -0,0 +1,148 @@ + + + + + 10.6. Client failover - Apache Qpid™ + + + + + + + + + + + + + +

+ + +

+ + + + + +

+ + +

10.6. Client failover
Prev	Chapter 10. High Availability	Next

As mentioned above, the clients need to be able to find the location of the active + virtualhost within the group.

Clients can do this using a static technique, for example , utilising the failover feature of the Qpid connection url + where the client has a list of all the nodes, and tries each node in sequence until it + discovers the node with the active virtualhost.

Another possibility is a dynamic technique utilising a proxy or Virtual IP (VIP). These + require other software and/or hardware and are outside the scope of this document.

Prev	Up	Next
10.5. Node Operations	Home	10.7. Disk space requirements

+ +

+ + + +

+ Apache Qpid, Messaging built on AMQP; Copyright © 2015 + The Apache Software Foundation; Licensed under + the Apache + License, Version 2.0; Apache Qpid, Qpid, Qpid Proton, + Proton, Apache, the Apache feather logo, and the Apache Qpid + project logo are trademarks of The Apache Software + Foundation; All other marks mentioned may be trademarks or + registered trademarks of their respective owners +

+ + http://git-wip-us.apache.org/repos/asf/qpid-site/blob/dc514b74/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-CreatingGroup.html ---------------------------------------------------------------------- diff --git a/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-CreatingGroup.html b/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-CreatingGroup.html new file mode 100644 index 0000000..220e331 --- /dev/null +++ b/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-CreatingGroup.html @@ -0,0 +1,182 @@ + + + + + 10.3. Creating a group - Apache Qpid™ + + + + + + + + + + + + + +

+ + +

+ + + + + +

+ + +

10.3. Creating a group
Prev	Chapter 10. High Availability	Next

10.3. Creating a group

This section describes how to create a group. At a high level, creating a group involves + first creating the first node standalone, then creating subsequent nodes referencing the first + node so the nodes can introduce themselves and gradually the group is built up.

A group is created through either Web Management or the REST API. These instructions + presume you are using Web Management. To illustrate the example it builds the group + illustrated in figure Figure 10.1, “3-node group deployed across three Brokers.”

Install a Broker on each machine that will be used to host the group. As messaging + clients will need to be able to connect to and authentication to all Brokers, it usually + makes sense to choose a common authentication mechanism e.g. Simple LDAP Authentication, + External with SSL client authentication or Kerberos.
Select one Broker instance to host the first node instance. This choice is an + arbitrary one. The node is special only whilst creating group. Once creation is + complete, all nodes will be considered equal.
Click the Add button on the Virtualhost Panel on the Broker + tab.
+
1. Give the Virtualhost node a unique name e.g. weather1. The + name must be unique within the group and unique to that Broker. It is best if the + node names are chosen from a different nomenclature than the machine names + themselves.
2. Choose BDB_HA and select New group +
3. Give the group a name e.g. weather. The group name must be + unique and will be the name also given to the virtualhost, so this is the name the + messaging clients will use in their connection url.
4. Give the address of this node. This is an address on this node's host that + will be used for replication purposes. The hostname must be + resolvable by all the other nodes in the group. This is separate from the address + used by messaging clients to connect to the Broker. It is usually best to choose a + symbolic name, rather than an IP address.
5. Now add the node addresses of all the other nodes that will form the group. In + our example we are building a three node group so we give the node addresses of + chaac:5000 and indra:5000.
6. Click Add to create the node. The virtualhost node will be created with the + virtualhost. As there is only one node at this stage, the role will be + master.
+
Figure 10.2. Creating 1st node in a group

+
Now move to the second Broker to be the group. Click the Add + button on the Virtualhost Panel on the Broker tab of the second Broker.
+
1. Give the Virtualhost node a unique name e.g. + weather2.
2. Choose BDB_HA and choose Existing group +
3. Give the details of the existing node. Following our + example, specify weather, weather1 and + thor:5000
4. Give the address of this node.
5. Click Add to create the node. The node will use the existing details to + contact it and introduce itself into the group. At this stage, the group will have + two nodes, with the second node in the replica role.
6. Repeat these steps until you have added all the nodes to the group.
+
Figure 10.3. Adding subsequent nodes to the group

+

The group is now formed and is ready for us. Looking at the virtualhost node of any of the + nodes shows a complete view of the whole group.

Figure 10.4. View of group from one node

Prev	Up	Next
10.2. High Availability Overview	Home	10.4. Behaviour of the Group

+ +

+ + + +

+ + http://git-wip-us.apache.org/repos/asf/qpid-site/blob/dc514b74/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-DiskSpace.html ---------------------------------------------------------------------- diff --git a/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-DiskSpace.html b/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-DiskSpace.html new file mode 100644 index 0000000..244ca7e --- /dev/null +++ b/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-DiskSpace.html @@ -0,0 +1,147 @@ + + + + + 10.7. Disk space requirements - Apache Qpid™ + + + + + + + + + + + + + +

+ + +

+ + + + + +

+ + +

10.7. Disk space requirements
Prev	Chapter 10. High Availability	Next

10.7. Disk space requirements

In the case where node in a group are down, the master must keep the data they are missing + for them to allow them to return to the replica role quickly.

By default, the master will retain up to 1hour of missed transactions. In a busy + production system, the disk space occupied could be considerable.

This setting is controlled by virtualhost context variable + je.rep.repStreamTimeout.

Prev	Up	Next
10.6. Client failover	Home	10.8. Network Requirements

+ +

+ + + +

+ + http://git-wip-us.apache.org/repos/asf/qpid-site/blob/dc514b74/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-Network-Requirements.html ---------------------------------------------------------------------- diff --git a/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-Network-Requirements.html b/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-Network-Requirements.html new file mode 100644 index 0000000..0ed9cd1 --- /dev/null +++ b/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-Network-Requirements.html @@ -0,0 +1,148 @@ + + + + + 10.8. Network Requirements - Apache Qpid™ + + + + + + + + + + + + + +

+ + +

+ + + + + +

+ + +

10.8. Network Requirements
Prev	Chapter 10. High Availability	Next

10.8. Network Requirements

The HA Cluster performance depends on the network bandwidth, its use by existing traffic, + and quality of service.

In order to achieve the best performance it is recommended to use a separate network + infrastructure for the Qpid HA Nodes which might include installation of dedicated network + hardware on Broker hosts, assigning a higher priority to replication ports, installing a group + in a separate network not impacted by any other traffic.

Prev	Up	Next
10.7. Disk space requirements	Home	10.9. Security

+ +

+ + + +

+ + http://git-wip-us.apache.org/repos/asf/qpid-site/blob/dc514b74/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-NodeOperations.html ---------------------------------------------------------------------- diff --git a/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-NodeOperations.html b/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-NodeOperations.html new file mode 100644 index 0000000..64520b9 --- /dev/null +++ b/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-NodeOperations.html @@ -0,0 +1,164 @@ + + + + + 10.5. Node Operations - Apache Qpid™ + + + + + + + + + + + + + +

+ + +

+ + + + + +

+ + +

10.5. Node Operations
Prev	Chapter 10. High Availability	Next

10.5. Node Operations

10.5.1. Lifecycle

Virtualhost nodes can be stopped, started and deleted.

Stop
Stopping a master node will cause the node to temporarily leave the group. Any + messaging clients will be disconnected and any in-flight transaction rollbacked. The + remaining nodes will elect a new master if quorum number of nodes still remains.
Stopping a replica node will cause the node to temporarily leave the group too. + Providing quorum still exists, the current master will continue without interruption. If + by leaving the group, quorum no longer exists, all the nodes will begin waiting, + disconnecting any messaging clients, and the virtualhost will become unavailable.
A stopped virtualhost node is still considered to be a member of the group.
Start
Starting a virtualhost node allows it to rejoin the group.
If the group already has a master, the node will catch up from the master and then + become a replica once it has done so.
If the group did not have quorum and so had no master, but the rejoining of this + node means quorum now exists, an election will take place. The node with the most up to + date transaction will become master unless influenced by the priority rules described + above.
Note
The length of time taken to catch up will depend on how long the node has been + stopped. The worst case is where the node has been stopped for more than one hour. In + this case, the master will perform an automated network restore. + This involves streaming all the data held by the master over to the replica. This + could take considerable time.
Delete
A virtualhost node can be deleted. Deleting a node permanently removes the node from + the group. The data stored locally is removed but this does not affect the data held by + the remainder of the group.
Note
The names of deleted virtualhost node cannot be reused within a group.

It is also possible to add nodes to an existing group using the procedure described + above.

10.5.2. Transfer Master

This operation allows the mastership to be moved from node to node. This is useful for + restoring a business as usual state after a failure.

When using this function, the following occurs.

The system first gives time for the chosen new master to become reasonable up to + date.
It then suspends transactions on the old master and allows the chosen node to + become up to date.
The suspended transactions are aborted and any messaging clients connected to the + old master are disconnected.
The chosen master becomes the new master. The old master becomes a replica.
Messaging clients reconnect the new master.

Prev	Up	Next
10.4. Behaviour of the Group	Home	10.6. Client failover

+ +

+ + + +

+ + http://git-wip-us.apache.org/repos/asf/qpid-site/blob/dc514b74/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-OverviewOfHA.html ---------------------------------------------------------------------- diff --git a/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-OverviewOfHA.html b/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-OverviewOfHA.html new file mode 100644 index 0000000..84c0cfe --- /dev/null +++ b/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-OverviewOfHA.html @@ -0,0 +1,169 @@ + + + + + 10.2. High Availability Overview - Apache Qpid™ + + + + + + + + + + + + + +

+ + +

+ + + + + +

+ + +

10.2. High Availability Overview
Prev	Chapter 10. High Availability	Next

10.2. High Availability Overview

The Broker provides a HA implementation offering an Active/Passive mode of operation. + When using HA, many instances of the Broker work together to form an high availability group of two or more nodes.

The remainder of this section now talks about the specifics of how HA is achieved in terms + of the concepts introduced earlier in this + book.

The Virtualhost is the unit of + replication. This means that any durable queues, exchanges, and bindings + belonging to that virtualhost, any persistent messages contained within + the queues and any attribute settings applied to the virtualhost itself are automatically + replicated to all nodes within the group.^[14]

It is the Virtualhost Nodes + (from different Broker instances) that join together to form a group. The virtualhost nodes + collectively to coordinate the group: they organise replication between the master and + replicas and conduct elections to determine who becomes the new master in the event of the old + failing.

When a virtualhost node is in the master role, the virtualhost + beneath it is available for messaging work. Any write operations sent to the virtualhost are + automatically replicated to all other nodes in group.

When a virtualhost node is in the replica role, the virtualhost + beneath it is always unavailable for message work. Any attempted connections to a virtualhost + in this state are automatically turned away, allowing a messaging client to discover where the + master currently resides. When in replica role, the node sole responsibility is to consume a + replication stream in order that it remains up to date with the master.

Messaging clients discover the active virtualhost.This can be achieved using a static + technique (for instance, a failover url (a feature of a Apache Qpid JMS Client for AMQP 0-9-1/0-10)), or a dynamic one + utilising some kind of proxy or virtual IP (VIP).

The figure that follows illustrates a group formed of three virtualhost nodes from three + separate Broker instances. A client is connected to the virtualhost node that is in the master + role. The two virtualhost nodes weather1 and weather3 + are replicas and are receiving a stream of updates.

Figure 10.1. 3-node group deployed across three Brokers.

Diagram showing a 3 node group deployed across three Brokers

Currently, the only virtualhost/virtualhost node type offering HA is BDB HA. Internally, + this leverages the HA capabilities of the Berkeley DB JE edition. BDB JE is an optional dependency of + the Broker.

Note

The HA solution from the Apache Qpid Broker for Java is incompatible with the HA solution offered by the CPP + Broker. It is not possible to co-locate Broker for Java and CPP Brokers within the same group.

^[14]Transient messages and messages on non-durable queues are not replicated.

Prev	Up	Next
Chapter 10. High Availability	Home	10.3. Creating a group

+ +

+ + + +

+ + http://git-wip-us.apache.org/repos/asf/qpid-site/blob/dc514b74/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-Reset-Group-Infomational.html ---------------------------------------------------------------------- diff --git a/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-Reset-Group-Infomational.html b/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-Reset-Group-Infomational.html new file mode 100644 index 0000000..f0643dc --- /dev/null +++ b/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-Reset-Group-Infomational.html @@ -0,0 +1,151 @@ + + + + + 10.11. Reset Group Information - Apache Qpid™ + + + + + + + + + + + + + +

+ + +

+ + + + + +

+ + +

10.11. Reset Group Information
Prev	Chapter 10. High Availability	Next

10.11. Reset Group Information

BDB JE internally stores details of the group within its database. There are some + circumstances when resetting this information is useful.

Copying data between environments (e.g. production to UAT)
Some disaster recovery situations where a group must be recreated on new + hardware

This is not an normal operation and is not usually required

The following command replaces the group table contained within the JE logs files with the + provided information.

Example 10.1. Resetting of replication group with DbResetRepGroup

java -cp je-5.0.104.jar com.sleepycat.je.rep.util.DbResetRepGroup -h path/to/jelogfiles
-groupName newgroupname -nodeName newnodename -nodeHostPort newhostname:5000

The modified log files can then by copied into + ${QPID_WORK}/<nodename>/config directory of a target Broker. Then + start the Broker, and add a BDB HA Virtualhost node specify the same group name, node name and + node address. You will then have a group with a single node, ready to start re-adding + additional nodes as described above.

Prev	Up	Next
10.10. Backups	Home	Chapter 11. Backup And Recovery

+ +

+ + + +

+ + http://git-wip-us.apache.org/repos/asf/qpid-site/blob/dc514b74/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-Security.html ---------------------------------------------------------------------- diff --git a/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-Security.html b/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-Security.html new file mode 100644 index 0000000..00ddd81 --- /dev/null +++ b/content/releases/qpid-java-6.1.6/java-broker/book/Java-Broker-High-Availability-Security.html @@ -0,0 +1,146 @@ + + + + + 10.9. Security - Apache Qpid™ + + + + + + + + + + + + + +

+ + +

+ + + + + +

+ + +

10.9. Security
Prev	Chapter 10. High Availability	Next

10.9. Security

The replication stream between the master and the replicas is insecure and can be + intercepted by anyone having access to the replication network.

In order to reduce the security risks the entire HA group is recommended to run in a + separate network protected from general access and/or utilise SSH-tunnels/IPsec.

Prev	Up	Next
10.8. Network Requirements	Home	10.10. Backups

+ +

+ + + +

+ + --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscribe@qpid.apache.org For additional commands, e-mail: commits-help@qpid.apache.org

Project

Messaging APIs

Servers and tools

Resources