From commits-return-43717-archive-asf-public=cust-asf.ponee.io@qpid.apache.org Thu Feb 8 22:15:00 2018 Return-Path: X-Original-To: archive-asf-public@eu.ponee.io Delivered-To: archive-asf-public@eu.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by mx-eu-01.ponee.io (Postfix) with ESMTP id 6B54B180677 for ; Thu, 8 Feb 2018 22:14:59 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 56768160C5F; Thu, 8 Feb 2018 21:14:59 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 0739F160C4A for ; Thu, 8 Feb 2018 22:14:56 +0100 (CET) Received: (qmail 89095 invoked by uid 500); 8 Feb 2018 21:14:51 -0000 Mailing-List: contact commits-help@qpid.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@qpid.apache.org Delivered-To: mailing list commits@qpid.apache.org Received: (qmail 88803 invoked by uid 99); 8 Feb 2018 21:14:51 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 08 Feb 2018 21:14:51 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id D415BF3539; Thu, 8 Feb 2018 21:14:49 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: orudyy@apache.org To: commits@qpid.apache.org Date: Thu, 08 Feb 2018 21:14:57 -0000 Message-Id: In-Reply-To: <271a997bf3864edb80636b74e3eba227@git.apache.org> References: <271a997bf3864edb80636b74e3eba227@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [09/32] qpid-site git commit: QPID-8094: Update site for Qpid Broker-J release 7.0.1 http://git-wip-us.apache.org/repos/asf/qpid-site/blob/42d76007/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Getting-Started-CommandLine.html.in ---------------------------------------------------------------------- diff --git a/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Getting-Started-CommandLine.html.in b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Getting-Started-CommandLine.html.in new file mode 100644 index 0000000..4863b20 --- /dev/null +++ b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Getting-Started-CommandLine.html.in @@ -0,0 +1,58 @@ +
3.5. Using the command line
Prev Chapter 3. Getting Started Next

3.5. Using the command line

The Broker understands a number of command line options which may be used to customise the configuration.

+ For additional details about the broker configuration and related command line arguments see + Chapter 5, Initial Configuration. + The broker is fully configurable via its Web Management Console, for details of this see + Section 6.2, “Web Management Console”. +

To see usage information for all command line options, use the --help option

bin/qpid-server --help
usage: Qpid [-cic <path>] [-h] [-icp <path>] [-mm] [-mmhttp <port>]
+       [-mmpass <password>] [-mmqv] [-os]
+       [-prop <name=value>] [-props <path>] [-sp <path>] [-st <type>] [-v]
+ -cic,--create-initial-config <path>                  create a copy of the
+                                                      initial config file,
+                                                      either to an
+                                                      optionally specified
+                                                      file path, or as
+                                                      initial-config.json
+                                                      in the current
+                                                      directory
+ -h,--help                                            print this message
+ -icp,--initial-config-path <path>                    set the location of
+                                                      initial JSON config
+                                                      to use when
+                                                      creating/overwriting
+                                                      a broker
+                                                      configuration store
+ -mm,--management-mode                                start broker in
+                                                      management mode,
+                                                      disabling the AMQP
+                                                      ports
+ -mmhttp,--management-mode-http-port <port>           override http
+                                                      management port in
+                                                      management mode
+ -mmpass,--management-mode-password <password>        Set the password for
+                                                      the management mode
+                                                      user mm_admin
+ -mmqv,--management-mode-quiesce-virtualhosts         make virtualhosts
+                                                      stay in the quiesced
+                                                      state during
+                                                      management mode.
+ -prop,--config-property <name=value>                 set a configuration
+                                                      property to use when
+                                                      resolving variables
+                                                      in the broker
+                                                      configuration store,
+                                                      with format
+                                                      "name=value"
+ -props,--system-properties-file <path>               set the location of
+                                                      initial properties
+                                                      file to set
+                                                      otherwise unset
+                                                      system properties
+ -sp,--store-path <path>                              use given
+                                                      configuration store
+                                                      location
+ -st,--store-type <type>                              use given broker
+                                                      configuration store
+                                                      type
+ -v,--version                                         print the version
+                                                      information and exit
+
Prev Up Next
3.4. Log file Home Chapter 4. Concepts
\ No newline at end of file http://git-wip-us.apache.org/repos/asf/qpid-site/blob/42d76007/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Getting-Started-Logging.html.in ---------------------------------------------------------------------- diff --git a/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Getting-Started-Logging.html.in b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Getting-Started-Logging.html.in new file mode 100644 index 0000000..bed757d --- /dev/null +++ b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Getting-Started-Logging.html.in @@ -0,0 +1,4 @@ +
3.4. Log file
Prev Chapter 3. Getting Started Next

3.4. Log file

The Broker writes a log file to record both details of its normal operation and any exceptional + conditions. By default the log file is written within the log subdirectory beneath the work directory + - $QPID_WORK/log/qpid.log (UNIX) and + %QPID_WORK%\log\qpid.log (Windows).

For details of how to control the logging, see Section 9.1, “Logging”

Prev Up Next
3.3. Starting/Stopping the broker on Unix Home 3.5. Using the command l ine
\ No newline at end of file http://git-wip-us.apache.org/repos/asf/qpid-site/blob/42d76007/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Getting-Started-Starting-Stopping-Unix.html.in ---------------------------------------------------------------------- diff --git a/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Getting-Started-Starting-Stopping-Unix.html.in b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Getting-Started-Starting-Stopping-Unix.html.in new file mode 100644 index 0000000..eb3ce70 --- /dev/null +++ b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Getting-Started-Starting-Stopping-Unix.html.in @@ -0,0 +1,13 @@ +
3.3. Starting/Stopping the broker on Unix
Prev Chapter 3. Getting Started Next

3.3. Starting/Stopping the broker on Unix

Firstly change to the installation directory used during the installation + and ensure that the QPID_WORK environment variable is set.

Now use the qpid-server script to start the server:

bin/qpid-server

Output similar to the following will be seen:

[Broker] BRK-1006 : Using configuration : /var/qpidwork/config.json
+[Broker] BRK-1001 : Startup : Version: 7.0.1 Build: exported
+[Broker] BRK-1010 : Platform : JVM : Oracle Corporation version: 1.8.0_144-b01 OS : Mac OS X version: 10.12.6 arch: x86_64 cores: 8
+[Broker] BRK-1011 : Maximum Memory : Heap : 518,979,584 bytes Direct : 1,610,612,736 bytes
+[Broker] BRK-1002 : Starting : Listening on TCP port 5672
+[Broker] MNG-1001 : Web Management Startup
+[Broker] MNG-1002 : Starting : HTTP : Listening on port 8080
+[Broker] MNG-1004 : Web Management Ready
+[Broker] BRK-1004 : Qpid Broker Ready

The BRK-1004 message confirms that the Broker is ready for work. The MNG-1002 and BRK-1002 confirm the ports on + which the Broker is listening (for HTTP management and AMQP respectively).

To stop the Broker, use Control-C from the controlling shell, use the + bin/qpid.stop script, use kill -TERM <pid>, or + the REST operation broker/shutdown.

Prev Up Next
3.2. Starting/Stopping the broker on Windows Home 3.4. Log file
\ No newline at end of file http://git-wip-us.apache.org/repos/asf/qpid-site/blob/42d76007/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Getting-Started-Starting-Stopping-Windows.html.in ---------------------------------------------------------------------- diff --git a/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Getting-Started-Starting-Stopping-Windows.html.in b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Getting-Started-Starting-Stopping-Windows.html.in new file mode 100644 index 0000000..7100546 --- /dev/null +++ b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Getting-Started-Starting-Stopping-Windows.html.in @@ -0,0 +1,13 @@ +
3.2. Starting/Stopping the broker on Windows
Prev Chapter 3. Getting Started Next

3.2. Starting/Stopping the broker on Windows

Firstly change to the installation directory used during the installation + and ensure that the QPID_WORK environment variable is set.

Now use the qpid-server.bat to start the server

bin\qpid-server.bat

Output similar to the following will be seen:

[Broker] BRK-1006 : Using configuration : C:\qpidwork\config.json
+[Broker] BRK-1001 : Startup : Version: 7.0.1 Build: 1478262
+[Broker] BRK-1010 : Platform : JVM : Oracle Corporation version: 1.8.0_144-b01 OS : Windows 7 version: 6.1 arch: x86 cores: 4
+[Broker] BRK-1011 : Maximum Memory : Heap : 518,979,584 bytes Direct : 1,610,612,736 bytes
+[Broker] BRK-1002 : Starting : Listening on TCP port 5672
+[Broker] MNG-1001 : Web Management Startup
+[Broker] MNG-1002 : Starting : HTTP : Listening on port 8080
+[Broker] MNG-1004 : Web Management Ready
+[Broker] BRK-1004 : Qpid Broker Ready

The BRK-1004 message confirms that the Broker is ready for work. The MNG-1002 and BRK-1002 confirm the ports on + which the Broker is listening (for HTTP management and AMQP respectively).

To stop the Broker, use Control-C from the controlling command prompt or + REST operation broker/shutdown. +

Prev Up Next
Chapter 3. Getting Started Home 3.3. Starting/Stopping the broker on Unix
\ No newline at end of file http://git-wip-us.apache.org/repos/asf/qpid-site/blob/42d76007/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Getting-Started.html.in ---------------------------------------------------------------------- diff --git a/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Getting-Started.html.in b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Getting-Started.html.in new file mode 100644 index 0000000..55bc68b --- /dev/null +++ b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Getting-Started.html.in @@ -0,0 +1,8 @@ +
Chapter 3. Getting Started
Prev   Next

Chapter 3. Getting Started

Table of Contents

3.1. Introduction
3.2. Starting/S topping the broker on Windows
3.3. Starting/Stopping the broker on Unix
3.4. Log file
3.5. Using the command line

3.1. Introduction

+ This section describes how to start and stop the Broker, and outlines the various command line options. +

+ For additional details about the broker configuration store and related command line arguments see + Chapter 5, Initial Configuration. + The broker is fully configurable via its Web Management Console, for details of this see + Section 6.2, “Web Management Console”. +

Prev   Next
2.6. Optional Dependencies Home 3.2. Starting/Stopping the broker on Windows
\ No newline at end of file http://git-wip-us.apache.org/repos/asf/qpid-site/blob/42d76007/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-Backup.html.in ---------------------------------------------------------------------- diff --git a/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-Backup.html.in b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-Backup.html.in new file mode 100644 index 0000000..7fe4630 --- /dev/null +++ b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-Backup.html.in @@ -0,0 +1,2 @@ +
10.10. Backups
Prev Chapter 10. High Availability Next

10.10. Backups

It is recommend to use the hot backup script to periodically backup every node in the + group. Section 11.2.2, “BDB-HA”.

Prev Up Next
10.9. Security Home 10.11. Reset Group Information
\ No newline at end of file http://git-wip-us.apache.org/repos/asf/qpid-site/blob/42d76007/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-Behaviour.html.in ---------------------------------------------------------------------- diff --git a/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-Behaviour.html.in b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-Behaviour.html.in new file mode 100644 index 0000000..de18708 --- /dev/null +++ b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-Behaviour.html.in @@ -0,0 +1,83 @@ +
10.4. Behaviour of the Group
Prev Chapter 10. High Availability Next

10.4. Behaviour of the Group

This section first describes the behaviour of the group in its default configuration. It + then goes on to talk about the various controls that are available to override it. It + describes the controls available that affect the durability of transactions and + the data consistency between the master and replicas and thus make trade offs between + performance and reliability.

10.4.1. Default Behaviour

Let's first look at the behaviour of a group in default configuration.

In the default configuration, for any messaging work to be done, there must be at least + quorum nodes present. This means for example, in a three node group, + this means there must be at least two nodes available.

When a messaging client sends a transaction, it can be assured that, before the control + returns back to his application after the commit call that the following is true:

  • At the master, the transaction is written to disk and OS level caches + are flushed meaning the data is on the storage device.

  • At least quorum minus 1 replicas, acknowledge the receipt of + transaction. The replicas will write the data to the storage device + sometime later.

If there were to be a master failure immediately after the transaction was committed, + the transaction would be held by at least quorum minus one replicas. For example, if we had + a group of three, then we would be assured that at least one replica held the + transaction.

In the event of a master failure, if quorum nodes remain, those nodes hold an election. + The nodes will elect master the node with the most recent transaction. If two or more nodes + have the most recent transaction the group makes an arbitrary choice. If quorum number of + nodes does not remain, the nodes cannot elect a new master and will wait until nodes rejoin. + You will see later that manual controls are available allow service to be restored from + fewer than quorum nodes and to influence which node gets elected in the event of a + tie.

Whenever a group has fewer than quorum nodes present, the virtualhost will be + unavailable and messaging connections will be refused. If quorum disappears at the very + moment a messaging client sends a transaction that transaction will fail.

You will have noticed the difference in the synchronization policies applied the master + and the replicas. The replicas send the acknowledgement back before the data is written to + disk. The master synchronously writes the transaction to storage. This is an example of a + trade off between durability and performance. We will see more about how to control this + trade off later.

10.4.2. Synchronization Policy

The synchronization policy dictates what a node must do when it + receives a transaction before it acknowledges that transaction to the rest of the + group.

The following options are available:

  • SYNC. The node must write the transaction to disk and flush + any OS level buffers before sending the acknowledgement. SYNC is offers the highest + durability but offers the least performance.

  • WRITE_NO_SYNC. The node must write the transaction to disk + before sending the acknowledgement. OS level buffers will be flush as some point + later. This typically provides an assurance against failure of the application but not + the operating system or hardware.

  • NO_SYNC. The node immediately sends the acknowledgement. The + transaction will be written and OS level buffers flushed as some point later. NO_SYNC + offers the highest performance but the lowest durability level. This synchronization + policy is sometimes known as commit to the network.

It is possible to assign a one policy to the master and a different policy to the + replicas. These are configured as attributes on the + virtualhost. By default the master uses SYNC and replicas use + NO_SYNC.

10.4.3. Node Priority

Node priority can be used to influence the behaviour of the election algorithm. It is + useful in the case were you want to favour some nodes over others. For instance, if you wish + to favour nodes located in a particular data centre over those in a remote site.

The following options are available:

  • Highest. Nodes with this priority will be more favoured. In + the event of two or more nodes having the most recent transaction, the node with this + priority will be elected master. If two or more nodes have this priority the algorithm + will make an arbitrary choice.

  • High. Nodes with this priority will be favoured but not as + much so as those with Highest.

  • Normal. This is default election priority.

  • Never. The node will never be elected even if the + node has the most recent transaction. The node will still keep up to date + with the replication stream and will still vote itself, but can just never be + elected.

+

Node priority is configured as an attribute on the + virtualhost node and can be changed at runtime and is effective immediately.

Important

Use of the Never priority can lead to transaction loss. For example, consider a group + of three where replica-2 is marked as Never. If a transaction were to arrive and it be + acknowledged only by Master and Replica-2, the transaction would succeed. Replica 1 is + running behind for some reason (perhaps a full-GC). If a Master failure were to occur at + that moment, the replicas would elect Replica-1 even though Replica-2 had the most recent + transaction.

Transaction loss is reported by message HA-1014.

10.4.4. Required Minimum Number Of Nodes

This controls the required minimum number of nodes to complete a transaction and to + elect a new master. By default, the required number of nodes is set to + Default (which signifies quorum).

It is possible to reduce the required minimum number of nodes. The rationale for doing + this is normally to temporarily restore service from fewer than quorum nodes following an + extraordinary failure.

For example, consider a group of three. If one node were to fail, as quorum still + remained, the system would continue work without any intervention. If the failing node were + the master, a new master would be elected.

What if a further node were to fail? Quorum no longer remains, and the remaining node + would just wait. It cannot elect itself master. What if we wanted to restore service from + just this one node?

In this case, Required Number of Nodes can be reduced to 1 on the remain node, allowing + the node to elect itself and service to be restored from the singleton. Required minimum + number of nodes is configured as an attribute on the + virtualhost node and can be changed at runtime and is effective immediately.

Important

The attribute must be used cautiously. Careless use will lead to lost transactions and + can lead to a split-brain in the event of a network partition. If used to temporarily restore + service from fewer than quorum nodes, it is imperative to revert it + to the Default value as the failed nodes are restored.

Transaction loss is reported by message HA-1014.

10.4.5. Allow to Operate Solo

This attribute only applies to groups containing exactly two nodes.

In a group of two, if a node were to fail then in default configuration work will cease + as quorum no longer exists. A single node cannot elect itself master.

The allow to operate solo flag allows a node in a two node group to elect itself master and + to operate sole. It is configured as an attribute on the + virtualhost node and can be changed at runtime and is effective immediately.

For example, consider a group of two where the master fails. Service will be interrupted + as the remaining node cannot elect itself master. To allow it to become master, apply the + allow to operate solo flag to it. It will elect itself master and work can continue, albeit + from one node.

Important

It is imperative not to allow the allow to operate solo flag to be set on both nodes at once. To + do so will mean, in the event of a network partition, a split-brain will + occur.

Transaction loss is reported by message HA-1014.

10.4.6. Maximum message size

+ Internally, BDB JE HA restricts the maximum size of replication stream records passed from the master + to the replica(s). This helps prevent DOS attacks. + If expected application maximum message size is greater than 5MB, the BDB JE setting + je.rep.maxMessageSize and Qpid context variable qpid.max_message_size + needs to be adjusted to reflect this in order to avoid running into the BDB HA JE limit. +

Prev Up Next
10.3. Creating a group Home 10.5. Node Operations
\ No newline at end of file http://git-wip-us.apache.org/repos/asf/qpid-site/blob/42d76007/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-ClientFailover.html.in ---------------------------------------------------------------------- diff --git a/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-ClientFailover.html.in b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-ClientFailover.html.in new file mode 100644 index 0000000..dcd9c7f --- /dev/null +++ b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-ClientFailover.html.in @@ -0,0 +1,5 @@ +
10.6. Client failover
Prev Chapter 10. High Availability Next

10.6. Client failover

As mentioned above, the clients need to be able to find the location of the active + virtualhost within the group.

Clients can do this using a static technique, for example , utilising the failover feature of the Apache Qpid + JMS and Apache Qpid JMS AMQP 0-x clients where the client has a list of all the nodes, and tries each node in + sequence until it discovers the node with the active virtualhost.

Another possibility is a dynamic technique utilising a proxy or Virtual IP (VIP). These + require other software and/or hardware and are outside the scope of this document.

Prev Up Next
10.5. Node Operations Home 10.7. Disk space requirements
\ No newline at end of file http://git-wip-us.apache.org/repos/asf/qpid-site/blob/42d76007/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-CreatingGroup.html.in ---------------------------------------------------------------------- diff --git a/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-CreatingGroup.html.in b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-CreatingGroup.html.in new file mode 100644 index 0000000..2f4885a --- /dev/null +++ b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-CreatingGroup.html.in @@ -0,0 +1,39 @@ +
10.3. Creating a group
Prev Chapter 10. High Availability Next

10.3. Creating a group

This section describes how to create a group. At a high level, creating a group involves + first creating the first node standalone, then creating subsequent nodes referencing the first + node so the nodes can introduce themselves and gradually the group is built up.

A group is created through either Web Management or the REST API. These instructions + presume you are using Web Management. To illustrate the example it builds the group + illustrated in figure Figure 10.1, “3-node group deployed across three Brokers.”

  1. Install a Broker on each machine that will be used to host the group. As messaging + clients will need to be able to connect to and authentication to all Brokers, it usually + makes sense to choose a common authentication mechanism e.g. Simple LDAP Authentication, + External with SSL client authentication or Kerberos.

  2. Select one Broker instance to host the first node instance. This choice is an + arbitrary one. The node is special only whilst creating group. Once creation is + complete, all nodes will be considered equal.

  3. Click the Add button on the Virtualhost Panel on the Broker + tab.

    +

    1. Give the Virtualhost node a unique name e.g. weather1. The + name must be unique within the group and unique to that Broker. It is best if the + node names are chosen from a different nomenclature than the machine names + themselves.

    2. Choose BDB_HA and select New group +

    3. Give the group a name e.g. weather. The group name must be + unique and will be the name also given to the virtualhost, so this is the name the + messaging clients will use in their connection url.

    4. Give the address of this node. This is an address on this node's host that + will be used for replication purposes. The hostname must be + resolvable by all the other nodes in the group. This is separate from the address + used by messaging clients to connect to the Broker. It is usually best to choose a + symbolic name, rather than an IP address.

    5. Now add the node addresses of all the other nodes that will form the group. In + our example we are building a three node group so we give the node addresses of + chaac:5000 and indra:5000.

    6. Click Add to create the node. The virtualhost node will be created with the + virtualhost. As there is only one node at this stage, the role will be + master.

    +

    Figure 10.2. Creating 1st node in a group

    Creating 1st node in a group


    +

  4. Now move to the second Broker to be the group. Click the Add + button on the Virtualhost Panel on the Broker tab of the second Broker.

    +

    1. Give the Virtualhost node a unique name e.g. + weather2.

    2. Choose BDB_HA and choose Existing group +

    3. Give the details of the existing node. Following our + example, specify weather, weather1 and + thor:5000

    4. Give the address of this node.

    5. Click Add to create the node. The node will use the existing details to + contact it and introduce itself into the group. At this stage, the group will have + two nodes, with the second node in the replica role.

    6. Repeat these steps until you have added all the nodes to the group.

    +

    Figure 10.3. Adding subsequent nodes to the group

    Adding subsequent nodes to the group


    +

The group is now formed and is ready for us. Looking at the virtualhost node of any of the + nodes shows a complete view of the whole group.

Figure 10.4. View of group from one node

View of group from one node


Prev Up Next
10.2. High Availability Overview Home 10.4. Behaviour of the Group
\ No newline at end of file http://git-wip-us.apache.org/repos/asf/qpid-site/blob/42d76007/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-DiskSpace.html.in ---------------------------------------------------------------------- diff --git a/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-DiskSpace.html.in b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-DiskSpace.html.in new file mode 100644 index 0000000..57d40a3 --- /dev/null +++ b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-DiskSpace.html.in @@ -0,0 +1,4 @@ +
10.7. Disk space requirements
Prev Chapter 10. High Availability Next

10.7. Disk space requirements

In the case where node in a group are down, the master must keep the data they are missing + for them to allow them to return to the replica role quickly.

By default, the master will retain up to 1hour of missed transactions. In a busy + production system, the disk space occupied could be considerable.

This setting is controlled by virtualhost context variable + je.rep.repStreamTimeout.

Prev Up Next
10.6. Client failover Home 10.8. Network Requirements
\ No newline at end of file http://git-wip-us.apache.org/repos/asf/qpid-site/blob/42d76007/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-Network-Requirements.html.in ---------------------------------------------------------------------- diff --git a/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-Network-Requirements.html.in b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-Network-Requirements.html.in new file mode 100644 index 0000000..f9c88df --- /dev/null +++ b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-Network-Requirements.html.in @@ -0,0 +1,5 @@ +
10.8. Network Requirements
Prev Chapter 10. High Availability Next

10.8. Network Requirements

The HA Cluster performance depends on the network bandwidth, its use by existing traffic, + and quality of service.

In order to achieve the best performance it is recommended to use a separate network + infrastructure for the Qpid HA Nodes which might include installation of dedicated network + hardware on Broker hosts, assigning a higher priority to replication ports, installing a group + in a separate network not impacted by any other traffic.

Prev Up Next
10.7. Disk space requirements Home 10.9. Security
\ No newline at end of file http://git-wip-us.apache.org/repos/asf/qpid-site/blob/42d76007/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-NodeOperations.html.in ---------------------------------------------------------------------- diff --git a/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-NodeOperations.html.in b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-NodeOperations.html.in new file mode 100644 index 0000000..34d7df7 --- /dev/null +++ b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-NodeOperations.html.in @@ -0,0 +1,21 @@ +
10.5. Node Operations
Prev Chapter 10. High Availability Next

10.5. Node Operations

10.5.1. Lifecycle

Virtualhost nodes can be stopped, started and deleted.

  • Stop

    Stopping a master node will cause the node to temporarily leave the group. Any + messaging clients will be disconnected and any in-flight transaction rollbacked. The + remaining nodes will elect a new master if quorum number of nodes still remains.

    Stopping a replica node will cause the node to temporarily leave the group too. + Providing quorum still exists, the current master will continue without interruption. If + by leaving the group, quorum no longer exists, all the nodes will begin waiting, + disconnecting any messaging clients, and the virtualhost will become unavailable.

    A stopped virtualhost node is still considered to be a member of the group.

  • Start

    Starting a virtualhost node allows it to rejoin the group.

    If the group already has a master, the node will catch up from the master and then + become a replica once it has done so.

    If the group did not have quorum and so had no master, but the rejoining of this + node means quorum now exists, an election will take place. The node with the most up to + date transaction will become master unless influenced by the priority rules described + above.

    Note

    The length of time taken to catch up will depend on how long the node has been + stopped. The worst case is where the node has been stopped for more than one hour. In + this case, the master will perform an automated network restore. + This involves streaming all the data held by the master over to the replica. This + could take considerable time.

  • Delete

    A virtualhost node can be deleted. Deleting a node permanently removes the node from + the group. The data stored locally is removed but this does not affect the data held by + the remainder of the group.

    Note

    The names of deleted virtualhost node cannot be reused within a group.

It is also possible to add nodes to an existing group using the procedure described + above.

10.5.2. Transfer Master

This operation allows the mastership to be moved from node to node. This is useful for + restoring a business as usual state after a failure.

When using this function, the following occurs.

  1. The system first gives time for the chosen new master to become reasonable up to + date.

  2. It then suspends transactions on the old master and allows the chosen node to + become up to date.

  3. The suspended transactions are aborted and any messaging clients connected to the + old master are disconnected.

  4. The chosen master becomes the new master. The old master becomes a replica.

  5. Messaging clients reconnect the new master.

Prev Up Next
10.4. Behaviour of the Group Home 10.6. Client failover
\ No newline at end of file http://git-wip-us.apache.org/repos/asf/qpid-site/blob/42d76007/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-OverviewOfHA.html.in ---------------------------------------------------------------------- diff --git a/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-OverviewOfHA.html.in b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-OverviewOfHA.html.in new file mode 100644 index 0000000..e91c929 --- /dev/null +++ b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-OverviewOfHA.html.in @@ -0,0 +1,25 @@ +
10.2. High Availability Overview
Prev Chapter 10. High Availability Next

10.2. High Availability Overview

The Broker provides a HA implementation offering an Active/Passive mode of operation. + When using HA, many instances of the Broker work together to form an high availability group of two or more nodes.

The remainder of this section now talks about the specifics of how HA is achieved in terms + of the concepts introduced earlier in this + book.

The Virtualhost is the unit of + replication. This means that any durable queues, exchanges, and bindings + belonging to that virtualhost, any persistent messages contained within + the queues and any attribute settings applied to the virtualhost itself are automatically + replicated to all nodes within the group.[11]

It is the Virtualhost Nodes + (from different Broker instances) that join together to form a group. The virtualhost nodes + collectively to coordinate the group: they organise replication between the master and + replicas and conduct elections to determine who becomes the new master in the event of the old + failing.

When a virtualhost node is in the master role, the virtualhost + beneath it is available for messaging work. Any write operations sent to the virtualhost are + automatically replicated to all other nodes in group.

When a virtualhost node is in the replica role, the virtualhost + beneath it is always unavailable for message work. Any attempted connections to a virtualhost + in this state are automatically turned away, allowing a messaging client to discover where the + master currently resides. When in replica role, the node sole responsibility is to consume a + replication stream in order that it remains up to date with the master.

Messaging clients discover the active virtualhost.This can be achieved using a static + technique (for instance, a failover url (a feature of the Apache Qpid JMS and Apache Qpid JMS AMQP 0-x clients), + or a dynamic one utilising some kind of proxy or virtual IP (VIP).

The figure that follows illustrates a group formed of three virtualhost nodes from three + separate Broker instances. A client is connected to the virtualhost node that is in the master + role. The two virtualhost nodes weather1 and weather3 + are replicas and are receiving a stream of updates.

Figure 10.1. 3-node group deployed across three Brokers.

Diagram showing a 3 node group deployed across three Brokers

Currently, the only virtualhost/virtualhost node type offering HA is BDB HA. Internally, + this leverages the HA capabilities of the Berkeley DB JE edition.

Note

The HA solution from the Apache Qpid Broker-J is incompatible with the HA solution offered by the CPP + Broker. It is not possible to co-locate Qpid Broker-J and CPP Brokers within the same group.


[11] Transient messages and messages on non-durable queues are not replicated.

Prev Up Next
Chapter 10. High Availability Ho me 10.3. Creating a group
\ No newline at end of file http://git-wip-us.apache.org/repos/asf/qpid-site/blob/42d76007/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-Reset-Group-Infomational.html.in ---------------------------------------------------------------------- diff --git a/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-Reset-Group-Infomational.html.in b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-Reset-Group-Infomational.html.in new file mode 100644 index 0000000..409723b --- /dev/null +++ b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-Reset-Group-Infomational.html.in @@ -0,0 +1,8 @@ +
10.11. Reset Group Information
Prev Chapter 10. High Availability Next

10.11. Reset Group Information

BDB JE internally stores details of the group within its database. There are some + circumstances when resetting this information is useful.

  • Copying data between environments (e.g. production to UAT)

  • Some disaster recovery situations where a group must be recreated on new + hardware

This is not an normal operation and is not usually required

The following command replaces the group table contained within the JE logs files with the + provided information.

Example 10.1. Resetting of replication group with DbResetRepGroup

java -cp je-7.4.5.jar com.sleepycat.je.rep.util.DbResetRepGroup -h path/to/jelogfiles
-groupName newgroupname -nodeName newnodename -nodeHostPort newhostname:5000


The modified log files can then by copied into + ${QPID_WORK}/<nodename>/config directory of a target Broker. Then + start the Broker, and add a BDB HA Virtualhost node specify the same group name, node name and + node address. You will then have a group with a single node, ready to start re-adding + additional nodes as described above.

Prev Up Next
10.10. Backups Home Chapter 11. Backup And Recovery
\ No newline at end of file http://git-wip-us.apache.org/repos/asf/qpid-site/blob/42d76007/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-Security.html.in ---------------------------------------------------------------------- diff --git a/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-Security.html.in b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-Security.html.in new file mode 100644 index 0000000..6443f01 --- /dev/null +++ b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability-Security.html.in @@ -0,0 +1,3 @@ +
10.9. Security
Prev Chapter 10. High Availability Next

10.9. Security

The replication stream between the master and the replicas is insecure and can be + intercepted by anyone having access to the replication network.

In order to reduce the security risks the entire HA group is recommended to run in a + separate network protected from general access and/or utilise SSH-tunnels/IPsec.

Prev Up Next
10.8. Network Requirements Home 10.10. Backups
\ No newline at end of file http://git-wip-us.apache.org/repos/asf/qpid-site/blob/42d76007/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability.html.in ---------------------------------------------------------------------- diff --git a/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability.html.in b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability.html.in new file mode 100644 index 0000000..a83fc39 --- /dev/null +++ b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-High-Availability.html.in @@ -0,0 +1,10 @@ +
Chapter 10. High Availability
Prev   Next

Chapter 10. High Availability

Table of Contents

10.1. General Introduction
10.2. High Availability Over view
10.3. Creating a group
10.4. Behaviour of the Group
10.4.1. Default Behaviour
10.4.2. Synchronization Policy
10.4.3. Node Priority
10.4.4. Required Minimum Number Of Node s
10.4.5. Allow to Operate Solo
10.4.6. Maximum message size
10.5. Node Operations
10.5.1. Lifecycle
10.5.2. Transfer Master
10.6. Client fai lover
10.7. Disk space requirements
10.8. Network Requirements
10.9. Security
10.10. Backups
10.11. Reset Group Information

10.1. General Introduction

The term High Availability (HA) usually refers to having a number of instances of a + service such as a Message Broker available so that should a service unexpectedly fail, or + requires to be shutdown for maintenance, users may quickly connect to another instance and + continue their work with minimal interruption. HA is one way to make a overall system more + resilient by eliminating a single point of failure from a system.

HA offerings are usually categorised as Active/Active or + Active/Passive. An Active/Active system is one where all + nodes within the group are usually available for use by clients all of the time. In an + Active/Passive system, one only node within the group is available for use by clients at any + one time, whilst the others are in some kind of standby state, awaiting to quickly step-in in + the event the active node becomes unavailable.

Prev   Next
9.11. Memory Home 10.2. High Availability Overview
\ No newline at end of file http://git-wip-us.apache.org/repos/asf/qpid-site/blob/42d76007/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Initial-Configuration-Configuration-Properties.html.in ---------------------------------------------------------------------- diff --git a/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Initial-Configuration-Configuration-Properties.html.in b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Initial-Configuration-Configuration-Properties.html.in new file mode 100644 index 0000000..a6d1316 --- /dev/null +++ b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Initial-Configuration-Configuration-Properties.html.in @@ -0,0 +1,24 @@ +
5.6. Customising Configuration using Configuration Properties
Prev Chapter 5. Initial Configuration Next

5.6. Customising Configuration using Configuration Properties

It is possible for 'Initial Configuration' (and Configuration Store) files to contain + ${properties} that can be resolved to String values at startup, allowing a degree of + customisation using a fixed file. Configuration Property values can be set either via + Java System Properties, or by specifying ConfigurationProperties on the broker command + line. If both are defined, System Property values take precedence.

The broker has the following set of core configuration properties, with the indicated + default values if not otherwise configured by the user:

Table 5.1. Base Configuration Properties

Name Description Value
qpid.amqp_port Port number used for the brokers default AMQP messaging port "5672"
qpid.http_port Port number used for the brokers default HTTP management port "8080"
qpid.home_dir Location of the broker installation directory, which contains + the 'lib' directory and the 'etc' directory often used to store + files such as group and ACL files. Defaults to the value set into the QPID_HOME system property if + it is set, or remains unset otherwise unless configured by the user. +
qpid.work_dir Location of the broker working directory, which might contain + the persistent message store and broker configuration store files. Defaults to the value set into the QPID_WORK system property if + it is set, or the 'work' subdirectory of the JVMs current working + directory.


+

Use of these core properties can be seen in the default 'Initial Configuration' example.

Configuration Properties can be set on the command line using the + -prop (or --configuration-property) + command line argument: 

+$ ./qpid-server -prop "qpid.amqp_port=10000" -prop "qpid.http_port=10001"
+

In the example above, property used to set the port number of the default AMQP port + is specified with the value 10000, overriding the default value of 5672, and similarly + the value 10001 is used to override the default HTTP port number of 8080. When using the + 'Initial Configuration' to initialise a new Configuration Store (either at first broker + startup, when requesting to overwrite the configuration store) these new values will be used for the + port numbers instead.

NOTE: When running the broker on Windows and starting it via the qpid-server.bat + file, the "name=value" argument MUST be quoted.

Prev Up Next
5.5. Configuration Store Type Home 5.7. Example of JSON 'Initial Configuration'
\ No newline at end of file http://git-wip-us.apache.org/repos/asf/qpid-site/blob/42d76007/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Initial-Configuration-Create-Initial-Config.html.in ---------------------------------------------------------------------- diff --git a/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Initial-Configuration-Create-Initial-Config.html.in b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Initial-Configuration-Create-Initial-Config.html.in new file mode 100644 index 0000000..cdf281d --- /dev/null +++ b/input/releases/qpid-broker-j-7.0.1/book/Java-Broker-Initial-Configuration-Create-Initial-Config.html.in @@ -0,0 +1,16 @@ +
5.4. Creating an 'Initial Configuration' JSON File
Prev Chapter 5. Initial Configuration Next

5.4. Creating an 'Initial Configuration' JSON File

It is possible to have the broker output its default internal 'Initial Configuration' + file to disk using the command line argument -cic (or + --create-initial-config). If the option is used without + providing a path, a file called initial-config.json will be created + in the current directory, or alternatively the file can be created at a specified + location: 

+$ ./qpid-server -cic ./initial-config.json
+

The 'Initial Configuration' JSON file shares a common format with the brokers JSON + Configuration Store implementation, so it is possible to use a Broker's Configuration + Store output as an initial configuration. Typically 'Initial Configuration' files would + not to contain IDs for the configured entities, so that IDs will be generated when the + configuration store is initialised and prevent use of the same IDs across multiple + brokers, however it may prove useful to include IDs if using the Memory Configuration Store Type.

It can be useful to use Configuration + Properties within 'Initial Configuration' files to allow a degree of + customisation with an otherwise fixed file.

For an example file, see Section 5.7, “Example of JSON 'Initial Configuration'” +

Prev Up Next
5.3. 'Initial Configuration' Location Home 5.5. Configuration Store Type
\ No newline at end of file --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscribe@qpid.apache.org For additional commands, e-mail: commits-help@qpid.apache.org