accumulo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mwa...@apache.org
Subject [accumulo-website] branch asf-site updated: Jekyll build from master:1fbf0a9
Date Tue, 17 Jul 2018 14:18:42 GMT
This is an automated email from the ASF dual-hosted git repository.

mwalch pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/accumulo-website.git


The following commit(s) were added to refs/heads/asf-site by this push:
     new d57a433  Jekyll build from master:1fbf0a9
d57a433 is described below

commit d57a43300042142115d1bd2c1e26ebab95ea1359
Author: Mike Walch <mwalch@apache.org>
AuthorDate: Tue Jul 17 10:18:02 2018 -0400

    Jekyll build from master:1fbf0a9
    
    Improved linking in Replication docs (#99)
---
 docs/2.0/administration/replication.html | 146 ++++++++++---------------------
 feed.xml                                 |   4 +-
 search_data.json                         |   2 +-
 3 files changed, 49 insertions(+), 103 deletions(-)

diff --git a/docs/2.0/administration/replication.html b/docs/2.0/administration/replication.html
index 1a1ebdf..946081c 100644
--- a/docs/2.0/administration/replication.html
+++ b/docs/2.0/administration/replication.html
@@ -387,7 +387,7 @@ into the following sections.</p>
 
 <p>Each system involved in replication (even the primary) needs a name that uniquely
 identifies it across all peers in the replication graph. This should be considered
-fixed for an instance, and set in <code class="highlighter-rouge">accumulo-site.xml</code>.</p>
+fixed for an instance, and set using <a href="/docs/2.0/administration/properties#replication_name">replication.name</a>
in <code class="highlighter-rouge">accumulo-site.xml</code>.</p>
 
 <div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span
class="nt">&lt;property&gt;</span>
     <span class="nt">&lt;name&gt;</span>replication.name<span class="nt">&lt;/name&gt;</span>
@@ -463,52 +463,47 @@ Monitor server, using the <code class="highlighter-rouge">Replication</code>
lin
 
 <h2 id="work-assignment">Work Assignment</h2>
 
-<p>Depending on the schema of a table, different implementations of the WorkAssigner
used could
-be configured. The implementation is controlled via the property <code class="highlighter-rouge">replication.work.assigner</code>
-and the full class name for the implementation. This can be configured via the shell or
-<code class="highlighter-rouge">accumulo-site.xml</code>.</p>
+<p>Depending on the schema of a table, different implementations of the <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-server-base/1.9.1/org/apache/accumulo/server/replication/WorkAssigner.html">WorkAssigner</a>
+used could be configured. The implementation is controlled via the property <a href="/docs/2.0/administration/properties#replication_work_assigner">replication.work.assigner</a>
+and the full class name for the implementation. This can be configured via the shell or <code
class="highlighter-rouge">accumulo-site.xml</code>.</p>
 
-<div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span
class="nt">&lt;property&gt;</span>
-    <span class="nt">&lt;name&gt;</span>replication.work.assigner<span
class="nt">&lt;/name&gt;</span>
-    <span class="nt">&lt;value&gt;</span>org.apache.accumulo.master.replication.SequentialWorkAssigner<span
class="nt">&lt;/value&gt;</span>
-    <span class="nt">&lt;description&gt;</span>Implementation used to
assign work for replication<span class="nt">&lt;/description&gt;</span>
-<span class="nt">&lt;/property&gt;</span>
-</code></pre></div></div>
-
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>root@accumulo_primary&gt;
config -t my_table -s replication.work.assigner=org.apache.accumulo.master.replication.SequentialWorkAssigner
-</code></pre></div></div>
+<p>Two implementations of <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-server-base/1.9.1/org/apache/accumulo/server/replication/WorkAssigner.html">WorkAssigner</a>
are provided:</p>
 
-<p>Two implementations are provided. By default, the <code class="highlighter-rouge">SequentialWorkAssigner</code>
is configured for an
-instance. The SequentialWorkAssigner ensures that, per peer and each remote identifier, each
WAL is
-replicated in the order in which they were created. This is sufficient to ensure that updates
to a table
-will be replayed in the correct order on the peer. This implementation has the downside of
only replicating
-a single WAL at a time.</p>
-
-<p>The second implementation, the <code class="highlighter-rouge">UnorderedWorkAssigner</code>
can be used to overcome the limitation
+<ol>
+  <li>
+    <p>The <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-master/1.9.1/org/apache/accumulo/master/replication/UnorderedWorkAssigner.html">UnorderedWorkAssigner</a>
can be used to overcome the limitation
 of only a single WAL being replicated to a target and peer at any time. Depending on the
table schema,
 it’s possible that multiple versions of the same Key with different values are infrequent
or nonexistent.
 In this case, parallel replication to a peer and target is possible without any downsides.
In the case
 where this implementation is used were column updates are frequent, it is possible that there
will be
 an inconsistency between the primary and the peer.</p>
+  </li>
+  <li>
+    <p>The <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-master/1.9.1/org/apache/accumulo/master/replication/SequentialWorkAssigner.html">SequentialWorkAssigner</a>
is configured for an
+instance. The SequentialWorkAssigner ensures that, per peer and each remote identifier, each
WAL is
+replicated in the order in which they were created. This is sufficient to ensure that updates
to a table
+will be replayed in the correct order on the peer. This implementation has the downside of
only replicating
+a single WAL at a time.</p>
+  </li>
+</ol>
 
 <h2 id="replicasystems">ReplicaSystems</h2>
 
-<p><code class="highlighter-rouge">ReplicaSystem</code> is the interface
which allows abstraction of replication of data
-to peers of various types. Presently, only an <code class="highlighter-rouge">AccumuloReplicaSystem</code>
is provided
-which will replicate data to another Accumulo instance. A <code class="highlighter-rouge">ReplicaSystem</code>
implementation
-is run inside of the TabletServer process, and can be configured as mentioned in the 
-<code class="highlighter-rouge">Instance Configuration</code> section of this
document. Theoretically, an implementation
-of this interface could send data to other filesystems, databases, etc.</p>
+<p><a href="https://static.javadoc.io/org.apache.accumulo/accumulo-server-base/1.9.1/org/apache/accumulo/server/replication/ReplicaSystem.html">ReplicaSystem</a>
is the interface which allows abstraction of replication of data
+to peers of various types. Presently, only an <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-server-base/1.9.1/org/apache/accumulo/server/replication/AccumuloReplicaSystem.html">AccumuloReplicaSystem</a>
is provided
+which will replicate data to another Accumulo instance. A <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-server-base/1.9.1/org/apache/accumulo/server/replication/ReplicaSystem.html">ReplicaSystem</a>
implementation
+is run inside of the TabletServer process, and can be configured as mentioned in <a href="/docs/2.0/administration/replication#instance-configuration">Instance
Configuration</a>
+section of this document. Theoretically, an implementation of this interface could send data
to other filesystems, databases, etc.</p>
 
 <h3 id="accumuloreplicasystem">AccumuloReplicaSystem</h3>
 
-<p>The <code class="highlighter-rouge">AccumuloReplicaSystem</code> uses
Thrift to communicate with a peer Accumulo instance
+<p>The <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-server-base/1.9.1/org/apache/accumulo/server/replication/AccumuloReplicaSystem.html">AccumuloReplicaSystem</a>
uses Thrift to communicate with a peer Accumulo instance
 and replicate the necessary data. The TabletServer running on the primary will communicate
 with the Master on the peer to request the address of a TabletServer on the peer which
 this TabletServer will use to replicate the data.</p>
 
 <p>The TabletServer on the primary will then replicate data in batches of a configurable
-size (<code class="highlighter-rouge">replication.max.unit.size</code>). The
TabletServer on the peer will report how many
+size (<a href="/docs/2.0/administration/properties#replication_max_unit_size">replication.max.unit.size</a>).
The TabletServer on the peer will report how many
 records were applied back to the primary, which will be used to record how many records
 were successfully replicated. The TabletServer on the primary will continue to replicate
 data in these batches until no more data can be read from the file.</p>
@@ -518,77 +513,28 @@ data in these batches until no more data can be read from the file.</p>
 <p>There are a number of configuration values that can be used to control how
 the implementation of various components operate.</p>
 
-<table>
-  <thead>
-    <tr>
-      <th>Property</th>
-      <th>Description</th>
-      <th>Default</th>
-    </tr>
-  </thead>
-  <tbody>
-    <tr>
-      <td>replication.max.work.queue</td>
-      <td>Maximum number of files queued for replication at one time</td>
-      <td>1000</td>
-    </tr>
-    <tr>
-      <td>replication.work.assignment.sleep</td>
-      <td>Time between invocations of the WorkAssigner</td>
-      <td>30s</td>
-    </tr>
-    <tr>
-      <td>replication.worker.threads</td>
-      <td>Size of threadpool used to replicate data to peers</td>
-      <td>4</td>
-    </tr>
-    <tr>
-      <td>replication.receipt.service.port</td>
-      <td>Thrift service port to listen for replication requests, can use ‘0’ for
a random port</td>
-      <td>10002</td>
-    </tr>
-    <tr>
-      <td>replication.work.attempts</td>
-      <td>Number of attempts to replicate to a peer before aborting the attempt</td>
-      <td>10</td>
-    </tr>
-    <tr>
-      <td>replication.receiver.min.threads</td>
-      <td>Minimum number of idle threads for handling incoming replication</td>
-      <td>1</td>
-    </tr>
-    <tr>
-      <td>replication.receiver.threadcheck.time</td>
-      <td>Time between attempting adjustments of thread pool for incoming replications</td>
-      <td>30s</td>
-    </tr>
-    <tr>
-      <td>replication.max.unit.size</td>
-      <td>Maximum amount of data to be replicated in one RPC</td>
-      <td>64M</td>
-    </tr>
-    <tr>
-      <td>replication.work.assigner</td>
-      <td>Work Assigner implementation</td>
-      <td>org.apache.accumulo.master.replication.SequentialWorkAssigner</td>
-    </tr>
-    <tr>
-      <td>tserver.replication.batchwriter.replayer.memory</td>
-      <td>Size of BatchWriter cache to use in applying replication requests</td>
-      <td>50M</td>
-    </tr>
-  </tbody>
-</table>
+<ul>
+  <li><a href="/docs/2.0/administration/properties#replication_max_work_queue">replication.max.work.queue</a>
- Maximum number of files queued for replication at one time</li>
+  <li><a href="/docs/2.0/administration/properties#replication_work_assignment_sleep">replication.work.assignment.sleep</a>
- Time between invocations of the <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-server-base/1.9.1/org/apache/accumulo/server/replication/WorkAssigner.html">WorkAssigner</a></li>
+  <li><a href="/docs/2.0/administration/properties#replication_worker_threads">replication.worker.threads</a>
- Size of threadpool used to replicate data to peers</li>
+  <li><a href="/docs/2.0/administration/properties#replication_receipt_service_port">replication.receipt.service.port</a>
- Thrift service port to listen for replication requests, can use ‘0’ for a random port</li>
+  <li><a href="/docs/2.0/administration/properties#replication_work_attempts">replication.work.attempts</a>
- Number of attempts to replicate to a peer before aborting the attempt</li>
+  <li><a href="/docs/2.0/administration/properties#replication_receiver_min_threads">replication.receiver.min.threads</a>
- Minimum number of idle threads for handling incoming replication</li>
+  <li><a href="/docs/2.0/administration/properties#replication_receiver_threadcheck_time">replication.receiver.threadcheck.time</a>
- Time between attempting adjustments of thread pool for incoming replications</li>
+  <li><a href="/docs/2.0/administration/properties#replication_max_unit_size">replication.max.unit.size</a>
- Maximum amount of data to be replicated in one RPC</li>
+  <li><a href="/docs/2.0/administration/properties#replication_work_assigner">replication.work.assigner</a>
- <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-server-base/1.9.1/org/apache/accumulo/server/replication/WorkAssigner.html">WorkAssigner</a>
implementation</li>
+  <li><a href="/docs/2.0/administration/properties#tserver_replication_batchwriter_replayer_memory">tserver.replication.batchwriter.replayer.memory</a>
- Size of BatchWriter cache to use in applying replication requests</li>
+</ul>
 
 <h2 id="example-practical-configuration">Example Practical Configuration</h2>
 
 <p>A real-life example is now provided to give concrete application of replication
configuration. This
 example is a two instance Accumulo system, one primary system and one peer system. They are
called
-primary and peer, respectively. Each system also have a table of the same name, “my_table”.
The instance
-name for each is also the same (primary and peer), and both have ZooKeeper hosts on a node
with a hostname
+<strong>primary</strong> and <strong>peer</strong>, respectively.
Each system also have a table of the same name, <code class="highlighter-rouge">my_table</code>.
The instance
+name for each is also the same (<code class="highlighter-rouge">primary</code>
and <code class="highlighter-rouge">peer</code>), and both have ZooKeeper hosts
on a node with a hostname
 with that name as well (primary:2181 and peer:2181).</p>
 
-<p>We want to configure these systems so that “my_table” on “primary” replicates
to “my_table” on “peer”.</p>
+<p>We want to configure these systems so that <code class="highlighter-rouge">my_table</code>
on <strong>primary</strong> replicates to <code class="highlighter-rouge">my_table</code>
on <strong>peer</strong>.</p>
 
 <h3 id="accumulo-sitexml">accumulo-site.xml</h3>
 
@@ -600,7 +546,6 @@ in replication together. In this example, we will use the names provided
in the
 <div class="language-xml highlighter-rouge"><div class="highlight"><pre class="highlight"><code><span
class="nt">&lt;property&gt;</span>
   <span class="nt">&lt;name&gt;</span>replication.name<span class="nt">&lt;/name&gt;</span>
   <span class="nt">&lt;value&gt;</span>primary<span class="nt">&lt;/value&gt;</span>
-  <span class="nt">&lt;description&gt;</span>Defines the unique name<span
class="nt">&lt;/description&gt;</span>
 <span class="nt">&lt;/property&gt;</span>
 </code></pre></div></div>
 
@@ -646,10 +591,10 @@ root@peer&gt; tables -l
 
 <h4 id="define-the-peer-as-a-replication-peer-to-the-primary">Define the Peer as a
replication peer to the Primary</h4>
 
-<p>We’re defining the instance with replication.name of ‘peer’ as a peer. We
provide the implementation of ReplicaSystem
-that we want to use, and the configuration for the AccumuloReplicaSystem. In this case, the
configuration is the Accumulo
-Instance name for ‘peer’ and the ZooKeeper quorum string. The configuration key is of
the form
-“replication.peer.$peer_name”.</p>
+<p>We’re defining the instance with <a href="/docs/2.0/administration/properties#replication_name">replication.name</a>
of <code class="highlighter-rouge">peer</code> as a peer. We provide the implementation
of <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-server-base/1.9.1/org/apache/accumulo/server/replication/ReplicaSystem.html">ReplicaSystem</a>
+that we want to use, and the configuration for the <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-server-base/1.9.1/org/apache/accumulo/server/replication/AccumuloReplicaSystem.html">AccumuloReplicaSystem</a>.
In this case, the configuration is the Accumulo
+Instance name for <code class="highlighter-rouge">peer</code> and the ZooKeeper
quorum string. The configuration key is of the form
+<code class="highlighter-rouge">replication.peer.$peer_name</code>.</p>
 
 <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>root@primary&gt;
config -s replication.peer.peer=org.apache.accumulo.tserver.replication.AccumuloReplicaSystem,peer,$peer_zk_quorum
 </code></pre></div></div>
@@ -666,8 +611,8 @@ root@primary&gt; config -s replication.peer.password.peer=peer
 <h4 id="enable-replication-on-the-table">Enable replication on the table</h4>
 
 <p>Now that we have defined the peer on the primary and provided the authentication
credentials, we need to configure
-our table with the implementation of ReplicaSystem we want to use to replicate to the peer.
In this case, our peer 
-is an Accumulo instance, so we want to use the AccumuloReplicaSystem.</p>
+our table with the implementation of <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-server-base/1.9.1/org/apache/accumulo/server/replication/ReplicaSystem.html">ReplicaSystem</a>
we want to use to replicate to the peer. In this case, our peer
+is an Accumulo instance, so we want to use the <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-server-base/1.9.1/org/apache/accumulo/server/replication/AccumuloReplicaSystem.html">AccumuloReplicaSystem</a>.</p>
 
 <p>The configuration for the AccumuloReplicaSystem is the table ID for the table on
the peer instance that we
 want to replicate into. Be sure to use the correct value for $peer_table_id. The configuration
key is of
@@ -806,6 +751,7 @@ are processed most quickly and pushed through the replication framework.</p>
 the WAL is fully replicated to all remote locations.</p>
 
 
+
     <div class="row" style="margin-top: 20px;">
       <div class="col-md-10"><strong>Find documentation for all releases in the
<a href="/docs-archive">archive</strong></div>
       <div class="col-md-2"><a class="pull-right" href="https://github.com/apache/accumulo-website/edit/master/_docs-2-0/administration/replication.md"
role="button"><i class="glyphicon glyphicon-pencil"></i> <small>Edit
this page</small></a></div>
diff --git a/feed.xml b/feed.xml
index 541d448..f2cfcae 100644
--- a/feed.xml
+++ b/feed.xml
@@ -6,8 +6,8 @@
 </description>
     <link>https://accumulo.apache.org/</link>
     <atom:link href="https://accumulo.apache.org/feed.xml" rel="self" type="application/rss+xml"/>
-    <pubDate>Wed, 11 Jul 2018 17:09:36 -0400</pubDate>
-    <lastBuildDate>Wed, 11 Jul 2018 17:09:36 -0400</lastBuildDate>
+    <pubDate>Tue, 17 Jul 2018 10:17:51 -0400</pubDate>
+    <lastBuildDate>Tue, 17 Jul 2018 10:17:51 -0400</lastBuildDate>
     <generator>Jekyll v3.7.3</generator>
     
     
diff --git a/search_data.json b/search_data.json
index 1e23cad..a3a6534 100644
--- a/search_data.json
+++ b/search_data.json
@@ -58,7 +58,7 @@
   
     "docs-2-0-administration-replication": {
       "title": "Replication",
-      "content"	 : "OverviewReplication is a feature of Accumulo which provides a mechanism
to automaticallycopy data to other systems, typically for the purpose of disaster recovery,high
availability, or geographic locality. It is best to consider this featureas a framework for
automatic replication instead of the ability to copy datafrom to another Accumulo instance
as copying to another Accumulo cluster isonly an implementation detail. The local Accumulo
cluster is hereby referredto a [...]
+      "content"	 : "OverviewReplication is a feature of Accumulo which provides a mechanism
to automaticallycopy data to other systems, typically for the purpose of disaster recovery,high
availability, or geographic locality. It is best to consider this featureas a framework for
automatic replication instead of the ability to copy datafrom to another Accumulo instance
as copying to another Accumulo cluster isonly an implementation detail. The local Accumulo
cluster is hereby referredto a [...]
       "url": " /docs/2.0/administration/replication",
       "categories": "administration"
     },


Mime
View raw message