accumulo-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
Subject [accumulo-website] branch master updated: Updated MapReduce docs to include new configuration (#135)
Date Wed, 28 Nov 2018 19:13:45 GMT
This is an automated email from the ASF dual-hosted git repository.

mwalch pushed a commit to branch master
in repository

The following commit(s) were added to refs/heads/master by this push:
     new 2bab009  Updated MapReduce docs to include new configuration (#135)
2bab009 is described below

commit 2bab009eda7c4786a1b01f2c2e223c91072c30fb
Author: Mike Walch <>
AuthorDate: Wed Nov 28 14:13:41 2018 -0500

    Updated MapReduce docs to include new configuration (#135)
 _docs-2/administration/ |  1 +
 _docs-2/development/    | 15 +++++++++++++++
 2 files changed, 16 insertions(+)

diff --git a/_docs-2/administration/ b/_docs-2/administration/
index 4ad5a00..18c5714 100644
--- a/_docs-2/administration/
+++ b/_docs-2/administration/
@@ -42,6 +42,7 @@ Below are some changes in 2.0 that you should be aware of:
     - `` for all Accumulo services (except monitor)
     - `` for Accumulo monitor
     - `` for Accumulo clients and commands
+* [New Hadoop configuration is required]({% durl development/mapreduce#configuration %})
when reading or writing to Accumulo using MapReduce.
 * Run the command `accumulo shell` to access the shell using configuration in `conf/`
 When your Accumulo 2.0 installation is properly configured, stop Accumulo 1.8/9 and start
Accumulo 2.0:
diff --git a/_docs-2/development/ b/_docs-2/development/
index 0adb649..ee6a93a 100644
--- a/_docs-2/development/
+++ b/_docs-2/development/
@@ -13,6 +13,21 @@ can be set via these two format classes to do the following:
 * Restrict the scan to a range of rows
 * Restrict the input to a subset of available columns
+## Configuration
+Since 2.0.0, Accumulo no longer has the same versions of dependencies (i.e Guava, etc) as
+When launching a MapReduce job that reads or writes to Accumulo, you should build a shaded
+with all of your dependencies and complete the following steps so YARN only includes Hadoop
+(and not all of Hadoop dependencies) when running your MapReduce job:
+1. Set `export HADOOP_USE_CLIENT_CLASSLOADER=true` in your environment before submitting
+   your job with `yarn` command.
+2. Set the following in your Job configuration.
+    ```java
+    job.getConfiguration().set("mapreduce.job.classloader", "true");
+    ```
 ## Mapper and Reducer classes
 To read from an Accumulo table create a Mapper with the following class

View raw message