accumulo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From mwa...@apache.org
Subject [accumulo-website] branch asf-site updated: Jekyll build from master:2bab009
Date Wed, 28 Nov 2018 19:15:39 GMT
This is an automated email from the ASF dual-hosted git repository.

mwalch pushed a commit to branch asf-site
in repository https://gitbox.apache.org/repos/asf/accumulo-website.git


The following commit(s) were added to refs/heads/asf-site by this push:
     new 8ae53ce  Jekyll build from master:2bab009
8ae53ce is described below

commit 8ae53ce5bbf8347e2334f3b29d7a8b031ebc12e3
Author: Mike Walch <mwalch@apache.org>
AuthorDate: Wed Nov 28 14:15:21 2018 -0500

    Jekyll build from master:2bab009
    
    Updated MapReduce docs to include new configuration (#135)
---
 docs/2.x/administration/upgrading.html |  1 +
 docs/2.x/development/mapreduce.html    | 19 +++++++++++++++++++
 feed.xml                               |  4 ++--
 search_data.json                       |  4 ++--
 4 files changed, 24 insertions(+), 4 deletions(-)

diff --git a/docs/2.x/administration/upgrading.html b/docs/2.x/administration/upgrading.html
index b4c711f..fc30597 100644
--- a/docs/2.x/administration/upgrading.html
+++ b/docs/2.x/administration/upgrading.html
@@ -479,6 +479,7 @@ distributions of Hadoop.</li>
       <li><code class="highlighter-rouge">log4j.properties</code> for Accumulo
clients and commands</li>
     </ul>
   </li>
+  <li><a href="/docs/2.x/development/mapreduce#configuration">New Hadoop configuration
is required</a> when reading or writing to Accumulo using MapReduce.</li>
   <li>Run the command <code class="highlighter-rouge">accumulo shell</code>
to access the shell using configuration in <code class="highlighter-rouge">conf/accumulo-client.properties</code></li>
 </ul>
 
diff --git a/docs/2.x/development/mapreduce.html b/docs/2.x/development/mapreduce.html
index d3c9e0c..76a43f0 100644
--- a/docs/2.x/development/mapreduce.html
+++ b/docs/2.x/development/mapreduce.html
@@ -439,6 +439,25 @@ can be set via these two format classes to do the following:</p>
   <li>Restrict the input to a subset of available columns</li>
 </ul>
 
+<h2 id="configuration">Configuration</h2>
+
+<p>Since 2.0.0, Accumulo no longer has the same versions of dependencies (i.e Guava,
etc) as Hadoop.
+When launching a MapReduce job that reads or writes to Accumulo, you should build a shaded
jar
+with all of your dependencies and complete the following steps so YARN only includes Hadoop
code
+(and not all of Hadoop dependencies) when running your MapReduce job:</p>
+
+<ol>
+  <li>
+    <p>Set <code class="highlighter-rouge">export HADOOP_USE_CLIENT_CLASSLOADER=true</code>
in your environment before submitting
+your job with <code class="highlighter-rouge">yarn</code> command.</p>
+  </li>
+  <li>
+    <p>Set the following in your Job configuration.</p>
+    <div class="language-java highlighter-rouge"><div class="highlight"><pre
class="highlight"><code> <span class="n">job</span><span class="o">.</span><span
class="na">getConfiguration</span><span class="o">().</span><span
class="na">set</span><span class="o">(</span><span class="s">"mapreduce.job.classloader"</span><span
class="o">,</span> <span class="s">"true"</span><span class="o">);</span>
+</code></pre></div>    </div>
+  </li>
+</ol>
+
 <h2 id="mapper-and-reducer-classes">Mapper and Reducer classes</h2>
 
 <p>To read from an Accumulo table create a Mapper with the following class
diff --git a/feed.xml b/feed.xml
index e166a64..41524c2 100644
--- a/feed.xml
+++ b/feed.xml
@@ -6,8 +6,8 @@
 </description>
     <link>https://accumulo.apache.org/</link>
     <atom:link href="https://accumulo.apache.org/feed.xml" rel="self" type="application/rss+xml"/>
-    <pubDate>Tue, 20 Nov 2018 09:51:13 -0500</pubDate>
-    <lastBuildDate>Tue, 20 Nov 2018 09:51:13 -0500</lastBuildDate>
+    <pubDate>Wed, 28 Nov 2018 14:15:11 -0500</pubDate>
+    <lastBuildDate>Wed, 28 Nov 2018 14:15:11 -0500</lastBuildDate>
     <generator>Jekyll v3.7.3</generator>
     
     
diff --git a/search_data.json b/search_data.json
index 31e1a86..7b96bc9 100644
--- a/search_data.json
+++ b/search_data.json
@@ -51,7 +51,7 @@
   
     "docs-2-x-administration-upgrading": {
       "title": "Upgrading Accumulo",
-      "content"	 : "Upgrading from 1.8/9 to 2.0Follow the steps below to upgrade your Accumulo
instance and client to 2.0.Upgrade Accumulo instanceIMPORTANT! Before upgrading to Accumulo
2.0, you will need to upgrade to Java 8 and Hadoop 3.x.Upgrading to Accumulo 2.0 is done by
stopping Accumulo 1.8/9 and starting Accumulo 2.0.Before stopping Accumulo 1.8/9, install
Accumulo 2.0 and configure it by following the 2.0 quick start.There are several changes to
scripts and configuration in 2. [...]
+      "content"	 : "Upgrading from 1.8/9 to 2.0Follow the steps below to upgrade your Accumulo
instance and client to 2.0.Upgrade Accumulo instanceIMPORTANT! Before upgrading to Accumulo
2.0, you will need to upgrade to Java 8 and Hadoop 3.x.Upgrading to Accumulo 2.0 is done by
stopping Accumulo 1.8/9 and starting Accumulo 2.0.Before stopping Accumulo 1.8/9, install
Accumulo 2.0 and configure it by following the 2.0 quick start.There are several changes to
scripts and configuration in 2. [...]
       "url": " /docs/2.x/administration/upgrading",
       "categories": "administration"
     },
@@ -107,7 +107,7 @@
   
     "docs-2-x-development-mapreduce": {
       "title": "MapReduce",
-      "content"	 : "Accumulo tables can be used as the source and destination of MapReduce
jobs. Touse an Accumulo table with a MapReduce job, configure the job parameters to usethe
AccumuloInputFormat and AccumuloOutputFormat. Accumulo specific parameterscan be set via these
two format classes to do the following:  Authenticate and provide user credentials for the
input  Restrict the scan to a range of rows  Restrict the input to a subset of available columnsMapper
and Reducer classesTo [...]
+      "content"	 : "Accumulo tables can be used as the source and destination of MapReduce
jobs. Touse an Accumulo table with a MapReduce job, configure the job parameters to usethe
AccumuloInputFormat and AccumuloOutputFormat. Accumulo specific parameterscan be set via these
two format classes to do the following:  Authenticate and provide user credentials for the
input  Restrict the scan to a range of rows  Restrict the input to a subset of available columnsConfigurationSince
2.0.0, Ac [...]
       "url": " /docs/2.x/development/mapreduce",
       "categories": "development"
     },


Mime
View raw message