accumulo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From ktur...@apache.org
Subject [accumulo-website] branch asf-site updated: Jekyll build from master:23ecff6
Date Sun, 24 Feb 2019 00:16:18 GMT
This is an automated email from the ASF dual-hosted git repository.

kturner pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/accumulo-website.git


The following commit(s) were added to refs/heads/asf-site by this push:
     new b5f4773  Jekyll build from master:23ecff6
b5f4773 is described below

commit b5f4773a6aac9757773be9fd73f28cfda3212e86
Author: Keith Turner <kturner@apache.org>
AuthorDate: Sat Feb 23 19:15:41 2019 -0500

    Jekyll build from master:23ecff6
    
    fixes #152 update compaction strat docs (#158)
---
 docs/2.x/administration/scan-executors.html       | 35 ++++++++++++++++-------
 docs/2.x/getting-started/table_configuration.html | 26 ++++++++++++-----
 feed.xml                                          |  4 +--
 redirects.json                                    |  2 +-
 search_data.json                                  |  4 +--
 5 files changed, 49 insertions(+), 22 deletions(-)

diff --git a/docs/2.x/administration/scan-executors.html b/docs/2.x/administration/scan-executors.html
index 3e54979..29b3088 100644
--- a/docs/2.x/administration/scan-executors.html
+++ b/docs/2.x/administration/scan-executors.html
@@ -533,21 +533,36 @@ table <code class="highlighter-rouge">tex</code>.</p>
 
 <div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>config
-s tserver.scan.executors.special.threads=8
 config -s tserver.scan.executors.special.prioritizer=org.apache.accumulo.core.spi.scan.HintScanPrioritizer
+config -s tserver.scan.executors.special.prioritizer.opts.priority.alpha=1
+config -s tserver.scan.executors.special.prioritizer.opts.priority.gamma=3
 createtable tex
 config -t tex -s table.scan.dispatcher=org.apache.accumulo.core.spi.scan.SimpleScanDispatcher
-config -t tex -s table.scan.dispatcher.opts.heed_hints=true
+config -t tex -s table.scan.dispatcher.opts.executor.alpha=special
+config -t tex -s table.scan.dispatcher.opts.executor.gamma=special
 </code></pre></div></div>
 
 <p>The <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-core/2.0.0-alpha-2/org/apache/accumulo/core/spi/scan/HintScanPrioritizer.html">HintScanPrioritizer</a>
honors
-hints of the form <code class="highlighter-rouge">priority=&lt;integer&gt;</code>
to prioritize scans, with lower integers
-resulting in a higher priority. The <code class="highlighter-rouge">SimpleScanDispatcher</code>,
which is the
-default dispatcher, supports the <code class="highlighter-rouge">heed_hints</code>
option. By default the
-<code class="highlighter-rouge">SimpleScanDispatcher</code> ignores hints, but
when <code class="highlighter-rouge">heed_hints</code> is set to <code class="highlighter-rouge">true</code>
it
-will honor hints of the form <code class="highlighter-rouge">executor=&lt;executor
name&gt;</code> when choosing an
-executor. After restarting tservers, the following command will start a scan
-that uses the executor <code class="highlighter-rouge">special</code> with a
priority of 3.</p>
-
-<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>scan
-t tex --execution-hints priority=3,executor=special
+hints of the form <code class="highlighter-rouge">priority=&lt;integer&gt;</code>
or <code class="highlighter-rouge">scan_type=&lt;type&gt;</code> to prioritize
+scans, with lower integers resulting in a higher priority.  When a hint
+specifies a scan type it is mapped to a priority based on the prioritizer
+configuration.</p>
+
+<p>The <code class="highlighter-rouge">SimpleScanDispatcher</code>, which
is the default dispatcher, supports
+<code class="highlighter-rouge">executor.&lt;type&gt;=&lt;executor&gt;</code>
options. When a scanner sets a hint of the form
+<code class="highlighter-rouge">scan_type=&lt;type&gt;</code> it will
use the executor configured for that type.</p>
+
+<p>After restarting tservers, the following command will start a scan that uses
+the executor <code class="highlighter-rouge">special</code> with a priority of
3.  The scan dispatcher maps the scan
+type <code class="highlighter-rouge">gamma</code> to the executor <code class="highlighter-rouge">special</code>.
 The prioritizer maps the scan type
+<code class="highlighter-rouge">gamma</code> to a priority of 3.</p>
+
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>scan
-t tex --execution-hints scan_type=gamma
+</code></pre></div></div>
+
+<p>The following command will start a scan that uses the executor <code class="highlighter-rouge">special</code>
with a
+priority of 1.</p>
+
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>scan
-t tex --execution-hints scan_type=alpha
 </code></pre></div></div>
 
 
diff --git a/docs/2.x/getting-started/table_configuration.html b/docs/2.x/getting-started/table_configuration.html
index 9574593..0fa7067 100644
--- a/docs/2.x/getting-started/table_configuration.html
+++ b/docs/2.x/getting-started/table_configuration.html
@@ -842,13 +842,25 @@ qualified class name.</p>
 
 <p>Accumulo provides a few classes that can be used as an alternative compaction strategy.
These classes are located in the 
 <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-tserver/2.0.0-alpha-2/org/apache/accumulo/tserver/compaction/package-summary.html">org.apache.accumulo.tserver.compaction</a>
package. <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-tserver/2.0.0-alpha-2/org/apache/accumulo/tserver/compaction/EverythingCompactionStrategy.html">EverythingCompactionStrategy</a>
-will simply compact all files. This is the strategy used by the user <code class="highlighter-rouge">compact</code>
command. <a href="https://static.javadoc.io/org.apache.accumulo/accumulo-tserver/2.0.0-alpha-2/org/apache/accumulo/tserver/compaction/SizeLimitCompactionStrategy.html">SizeLimitCompactionStrategy</a>
compacts files no bigger than the limit set in the property <code class="highlighter-rouge">table.majc.compaction.strategy.opts.sizeLimit</code>.</p>
-
-<p><a href="https://static.javadoc.io/org.apache.accumulo/accumulo-tserver/2.0.0-alpha-2/org/apache/accumulo/tserver/compaction/TwoTierCompactionStrategy.html">TwoTierCompactionStrategy</a>
is a hybrid compaction strategy that supports two types of compression. If the total size
of
-files being compacted is larger than <code class="highlighter-rouge">table.majc.compaction.strategy.opts.file.large.compress.threshold</code>
than a larger 
-compression type will be used. The larger compression type is specified in <code class="highlighter-rouge">table.majc.compaction.strategy.opts.file.large.compress.type</code>.

-Otherwise, the configured table compression will be used. To use this strategy with minor
compactions set <a href="/docs/2.x/configuration/server-properties#table_file_compress_type">table.file.compress.type</a>
to <code class="highlighter-rouge">snappy</code> 
-and set a different compress type in <code class="highlighter-rouge">table.majc.compaction.strategy.opts.file.large.compress.type</code>
for larger files.</p>
+will simply compact all files. This is the strategy used by the user <code class="highlighter-rouge">compact</code>
command.</p>
+
+<p><a href="https://static.javadoc.io/org.apache.accumulo/accumulo-tserver/2.0.0-alpha-2/org/apache/accumulo/tserver/compaction/strategies/BasicCompactionStrategy.html">BasicCompactionStrategy</a>
is
+a compaction strategy that supports a few options based on file size.  It
+supports filtering out large files from ever being included in a compaction.
+It also supports using a different compression algorithm for larger files.
+This allows frequent compactions of smaller files to use a fast algorithm and
+infrequent compactions of more data to use a slower algorithm.  Using this may
+enable an increase in throughput w/o using a lot more space.</p>
+
+<p>The following shell command configures a table to use snappy for small files,
+gzip for files over 100M, and avoid compacting any file larger than 250M.</p>
+
+<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>config
-t myTable -s table.file.compress.type=snappy
+config -t myTable -s table.majc.compaction.strategy=org.apache.accumulo.tserver.compaction.strategies.BasicCompactionStrategy
+config -t myTable -s table.majc.compaction.strategy.opts.filter.size=250M
+config -t myTable -s table.majc.compaction.strategy.opts.large.compress.threshold=100M
+config -t myTable -s table.majc.compaction.strategy.opts.large.compress.type=gzip
+</code></pre></div></div>
 
 <h2 id="pre-splitting-tables">Pre-splitting tables</h2>
 
diff --git a/feed.xml b/feed.xml
index 4fb8bea..361a34a 100644
--- a/feed.xml
+++ b/feed.xml
@@ -6,8 +6,8 @@
 </description>
     <link>https://accumulo.apache.org/</link>
     <atom:link href="https://accumulo.apache.org/feed.xml" rel="self" type="application/rss+xml"/>
-    <pubDate>Fri, 08 Feb 2019 10:23:11 -0500</pubDate>
-    <lastBuildDate>Fri, 08 Feb 2019 10:23:11 -0500</lastBuildDate>
+    <pubDate>Sat, 23 Feb 2019 19:15:36 -0500</pubDate>
+    <lastBuildDate>Sat, 23 Feb 2019 19:15:36 -0500</lastBuildDate>
     <generator>Jekyll v3.7.3</generator>
     
     
diff --git a/redirects.json b/redirects.json
index 9d051e4..4db7434 100644
--- a/redirects.json
+++ b/redirects.json
@@ -1 +1 @@
-{"/release_notes/1.5.1.html":"https://accumulo.apache.org/release/accumulo-1.5.1/","/release_notes/1.6.0.html":"https://accumulo.apache.org/release/accumulo-1.6.0/","/release_notes/1.6.1.html":"https://accumulo.apache.org/release/accumulo-1.6.1/","/release_notes/1.6.2.html":"https://accumulo.apache.org/release/accumulo-1.6.2/","/release_notes/1.7.0.html":"https://accumulo.apache.org/release/accumulo-1.7.0/","/release_notes/1.5.3.html":"https://accumulo.apache.org/release/accumulo-1.5.3/"
[...]
\ No newline at end of file
+{"/release_notes/1.5.1.html":"https://accumulo.apache.org/release/accumulo-1.5.1/","/release_notes/1.6.0.html":"https://accumulo.apache.org/release/accumulo-1.6.0/","/release_notes/1.6.1.html":"https://accumulo.apache.org/release/accumulo-1.6.1/","/release_notes/1.6.2.html":"https://accumulo.apache.org/release/accumulo-1.6.2/","/release_notes/1.7.0.html":"https://accumulo.apache.org/release/accumulo-1.7.0/","/release_notes/1.5.3.html":"https://accumulo.apache.org/release/accumulo-1.5.3/"
[...]
\ No newline at end of file
diff --git a/search_data.json b/search_data.json
index 6be6a16..0385f83 100644
--- a/search_data.json
+++ b/search_data.json
@@ -44,7 +44,7 @@
   
     "docs-2-x-administration-scan-executors": {
       "title": "Scan Executors",
-      "content"	 : "Accumulo scans operate by repeatedly fetching batches of data from a
tabletserver.  On the tablet server side, a thread pool fetches batches.In Java threads pools
are called executors.  By default, a single executor pertablet server handles all scans in
FIFO order.  For some workloads, the singleFIFO executor is suboptimal.  For example, consider
many unimportant scansreading lots of data mixed with a few important scans reading small
amounts ofdata.  The long scans n [...]
+      "content"	 : "Accumulo scans operate by repeatedly fetching batches of data from a
tabletserver.  On the tablet server side, a thread pool fetches batches.In Java threads pools
are called executors.  By default, a single executor pertablet server handles all scans in
FIFO order.  For some workloads, the singleFIFO executor is suboptimal.  For example, consider
many unimportant scansreading lots of data mixed with a few important scans reading small
amounts ofdata.  The long scans n [...]
       "url": " /docs/2.x/administration/scan-executors",
       "categories": "administration"
     },
@@ -177,7 +177,7 @@
   
     "docs-2-x-getting-started-table-configuration": {
       "title": "Table Configuration",
-      "content"	 : "Accumulo tables have a few options that can be configured to alter the
defaultbehavior of Accumulo as well as improve performance based on the data stored.These
include locality groups, constraints, bloom filters, iterators, and blockcache.  See the server
properties documentation for a complete list of availableconfiguration options.Locality GroupsAccumulo
supports storing sets of column families separately on disk to allowclients to efficiently
scan over columns tha [...]
+      "content"	 : "Accumulo tables have a few options that can be configured to alter the
defaultbehavior of Accumulo as well as improve performance based on the data stored.These
include locality groups, constraints, bloom filters, iterators, and blockcache.  See the server
properties documentation for a complete list of availableconfiguration options.Locality GroupsAccumulo
supports storing sets of column families separately on disk to allowclients to efficiently
scan over columns tha [...]
       "url": " /docs/2.x/getting-started/table_configuration",
       "categories": "getting-started"
     },


Mime
View raw message