Return-Path: X-Original-To: archive-asf-public-internal@cust-asf2.ponee.io Delivered-To: archive-asf-public-internal@cust-asf2.ponee.io Received: from cust-asf.ponee.io (cust-asf.ponee.io [163.172.22.183]) by cust-asf2.ponee.io (Postfix) with ESMTP id 8768A200BCE for ; Fri, 2 Dec 2016 23:24:48 +0100 (CET) Received: by cust-asf.ponee.io (Postfix) id 861AD160B24; Fri, 2 Dec 2016 22:24:48 +0000 (UTC) Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by cust-asf.ponee.io (Postfix) with SMTP id 8304C160B08 for ; Fri, 2 Dec 2016 23:24:47 +0100 (CET) Received: (qmail 46880 invoked by uid 500); 2 Dec 2016 22:24:46 -0000 Mailing-List: contact commits-help@activemq.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@activemq.apache.org Delivered-To: mailing list commits@activemq.apache.org Received: (qmail 46871 invoked by uid 99); 2 Dec 2016 22:24:46 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd1-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 02 Dec 2016 22:24:46 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd1-us-west.apache.org (ASF Mail Server at spamd1-us-west.apache.org) with ESMTP id 46F5AC0480 for ; Fri, 2 Dec 2016 22:24:46 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd1-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: -1.199 X-Spam-Level: X-Spam-Status: No, score=-1.199 tagged_above=-999 required=6.31 tests=[KAM_ASCII_DIVIDERS=0.8, KAM_LAZY_DOMAIN_SECURITY=1, RP_MATCHES_RCVD=-2.999] autolearn=disabled Received: from mx1-lw-us.apache.org ([10.40.0.8]) by localhost (spamd1-us-west.apache.org [10.40.0.7]) (amavisd-new, port 10024) with ESMTP id nC59BHIRgPit for ; Fri, 2 Dec 2016 22:24:45 +0000 (UTC) Received: from mailrelay1-us-west.apache.org (mailrelay1-us-west.apache.org [209.188.14.139]) by mx1-lw-us.apache.org (ASF Mail Server at mx1-lw-us.apache.org) with ESMTP id B9B285F1F0 for ; Fri, 2 Dec 2016 22:24:44 +0000 (UTC) Received: from svn01-us-west.apache.org (svn.apache.org [10.41.0.6]) by mailrelay1-us-west.apache.org (ASF Mail Server at mailrelay1-us-west.apache.org) with ESMTP id 06181E0250 for ; Fri, 2 Dec 2016 22:24:43 +0000 (UTC) Received: from svn01-us-west.apache.org (localhost [127.0.0.1]) by svn01-us-west.apache.org (ASF Mail Server at svn01-us-west.apache.org) with ESMTP id B13533A0332 for ; Fri, 2 Dec 2016 22:24:43 +0000 (UTC) Content-Type: text/plain; charset="utf-8" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit Subject: svn commit: r1002074 - in /websites/production/activemq/content: cache/main.pageCache kahadb-replication-experimental.html Date: Fri, 02 Dec 2016 22:24:43 -0000 To: commits@activemq.apache.org From: buildbot@apache.org X-Mailer: svnmailer-1.0.9 Message-Id: <20161202222443.B13533A0332@svn01-us-west.apache.org> archived-at: Fri, 02 Dec 2016 22:24:48 -0000 Author: buildbot Date: Fri Dec 2 22:24:42 2016 New Revision: 1002074 Log: Production update by buildbot for activemq Modified: websites/production/activemq/content/cache/main.pageCache websites/production/activemq/content/kahadb-replication-experimental.html Modified: websites/production/activemq/content/cache/main.pageCache ============================================================================== Binary files - no diff available. Modified: websites/production/activemq/content/kahadb-replication-experimental.html ============================================================================== --- websites/production/activemq/content/kahadb-replication-experimental.html (original) +++ websites/production/activemq/content/kahadb-replication-experimental.html Fri Dec 2 22:24:42 2016 @@ -83,65 +83,10 @@ -

Note

-

This is under review - and not currently supported. See Replicated LevelDB Store for the successor.

-
- - -

Overview

- -

The new KahaDB store supports a very fast and flexible replication system. It features:

- -
  • Journal level replication (The translates into lower overhead to the master to replicate records).
  • Support for multiple slaves.
  • Support to dynamically add slaves at runtime.
  • Uses multiple concurrent data transfer sessions to do an initial slave synchronization.
  • Big slave synchronizations can be resumed so synchronization progress is not lost if a slave is restarted.
  • A configurable minimum number of replicas allows you to pause processing until the data has been guaranteed to be replicated enough times.
- - - -

Master Election

- -

KahaDB supports a pluggable Master Election algorithm but the only current implementation is one based on ZooKeeper.

- -

ZooKeeper is used to implement the master election algorithm. ZooKeeper is a very fast, replicated, in memory database with features that make it easy to implement cluster control algorithms. It is an Apache project which you can freely download. You must installed and have at least one ZooKeeper server running before setting up a KahaDB Master Slave configuration.

- -

Configuring a Broker:

- -

The ActiveMQ binary distribution includes a KahaDB HA broker configuration at $ACTIVEMQ_HOME/conf/ha.xml.

- -

It it setup to look for a ZooKeeper 3.0.0 server on localhost at port 2181. Edit the configuration if this is not where you are running your ZooKeeper server.

- -

Start the configuation up by running:

-
-
-prompt> $ACTIVEMQ_HOME/bin/activemq xbean:ha.xml
+

Note

This is under review - and not currently supported.

Overview

The new KahaDB store supports a very fast and flexible replication system. It features:

  • Journal level replication (The translates into lower overhead to the master to replicate records).
  • Support for multiple slaves.
  • Support to dynamically add slaves at runtime.
  • Uses multiple concurrent data transfer sessions to do an initial slave synchronization.
  • Big slave synchronizations can be resumed so synchronization progress is not lost if a slave is restarted.
  • A configurable minimum number of replicas allows you to pause proc essing until the data has been guaranteed to be replicated enough times.

Master Election

KahaDB supports a pluggable Master Election algorithm but the only current implementation is one based on ZooKeeper.

ZooKeeper is used to implement the master election algorithm. ZooKeeper is a very fast, replicated, in memory database with features that make it easy to implement cluster control algorithms. It is an Apache project which you can freely download. You must installed and have at least one ZooKeeper server running before setting up a KahaDB Master Slave configuration.

Configuring a Broker:

The ActiveMQ binary distribution includes a KahaDB HA broker configura tion at $ACTIVEMQ_HOME/conf/ha.xml.

It it setup to look for a ZooKeeper 3.0.0 server on localhost at port 2181. Edit the configuration if this is not where you are running your ZooKeeper server.

Start the configuation up by running:

+
prompt> $ACTIVEMQ_HOME/bin/activemq xbean:ha.xml
 
-
- -

The actual contents of the configuration file follows:

- -
Error formatting macro: snippet: java.lang.IndexOutOfBoundsException: Index: 20, Size: 20
- -

Understanding the kahadbReplication XML element

- -

The brokerURI Attribute

- -

Notice that the the brokerURI attribute points at another broker configuration file. The ha-broker.xml contains the actual broker configuration that the broker uses when the node take over as master. The ha-broker.xml configuration file is a standard broker configuration except in these aspects:

- -
  • It MUST set the start="false" attribute on the broker element.
  • It MUST not configure a persistenceAdapter.
- - -

The above rules allows the replication system to inject the replicated KahaDB store into the Master when it's starting up.

- -

The minimumReplicas Attribute

- -

The minimumReplicas specifies how many copies of the database are required before synchronous update operations are deemed successful. Setting this to 0 allows a broker to continue operating even if there are no slaves attached. If the value is set to 1 or greater, and there are no slaves attached, the brokers persistent message processing will be suspended until the minimum number of slaves are attached and the data synchronized.

- - -

The uri Attribute

- -

The uri attribute should always be configured with a kdbr:// based URI. KDBR stands for 'KahaDB Replication' and this is the replication protocol used between the masters and the slaves. The master binds the specified port with slaves subsequently connect to and establish replication sessions. The host name in the uri MUST get updated to the actual machine's host name since this is also used to identify the nodes in the cluster.

- -

The directory Attribute

- -

This is the data directory where the KahaDB will store it's persistence files.

+

The actual contents of the configuration file follows:

Error formatting macro: snippet: java.lang.IndexOutOfBoundsException: Index: 20, Size: 20

Understanding the kahadbReplication XML element

The brokerURI Attribute

Notice that the the brokerURI attribute points at another broker configuration file. The ha-broker.xml contains the actual broker configuration that the broker uses when the node take over as master. The ha-broker.xml configuration file is a standard broker configuration except in these aspects:

  • It MUST set the start="false" attribute on the broker element.
  • It MUST not configure a persistenceAdapter.

The above rules allows the replication system to inject the replicated KahaDB store into the Master when it's starti ng up.

The minimumReplicas Attribute

The minimumReplicas specifies how many copies of the database are required before synchronous update operations are deemed successful. Setting this to 0 allows a broker to continue operating even if there are no slaves attached. If the value is set to 1 or greater, and there are no slaves attached, the brokers persistent message processing will be suspended until the minimum number of slaves are attached and the data synchronized.

The uri Attribute

The uri attribute should always be configured with a kdbr:// based URI. KDBR stands for 'KahaDB Replication' and this is the replication protocol used between the masters and the slaves. The master binds the specified port with slaves subsequently connect to and establish replication sessions. The host name in the uri MUST get updated to the actual machine's host name since this is also used to identify the nodes in the cluster.

The directory Attribute

This is the data directory where the KahaDB will store it's persistence files.