From commits-return-22365-archive-asf-public=cust-asf.ponee.io@accumulo.apache.org Wed Nov 28 20:15:41 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id 4B606180658 for ; Wed, 28 Nov 2018 20:15:41 +0100 (CET) Received: (qmail 35214 invoked by uid 500); 28 Nov 2018 19:15:40 -0000 Mailing-List: contact commits-help@accumulo.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@accumulo.apache.org Delivered-To: mailing list commits@accumulo.apache.org Received: (qmail 35205 invoked by uid 99); 28 Nov 2018 19:15:40 -0000 Received: from ec2-52-202-80-70.compute-1.amazonaws.com (HELO gitbox.apache.org) (52.202.80.70) by apache.org (qpsmtpd/0.29) with ESMTP; Wed, 28 Nov 2018 19:15:40 +0000 Received: by gitbox.apache.org (ASF Mail Server at gitbox.apache.org, from userid 33) id C8DBA81E23; Wed, 28 Nov 2018 19:15:39 +0000 (UTC) Date: Wed, 28 Nov 2018 19:15:39 +0000 To: "commits@accumulo.apache.org" Subject: [accumulo-website] branch asf-site updated: Jekyll build from master:2bab009 MIME-Version: 1.0 Content-Type: text/plain; charset=utf-8 Content-Transfer-Encoding: 8bit Message-ID: <154343253974.14330.16623239680513602549@gitbox.apache.org> From: mwalch@apache.org X-Git-Host: gitbox.apache.org X-Git-Repo: accumulo-website X-Git-Refname: refs/heads/asf-site X-Git-Reftype: branch X-Git-Oldrev: f488ba6ebdec55d3de0fd7766b68f56bbcaf0cb9 X-Git-Newrev: 8ae53ce5bbf8347e2334f3b29d7a8b031ebc12e3 X-Git-Rev: 8ae53ce5bbf8347e2334f3b29d7a8b031ebc12e3 X-Git-NotificationType: ref_changed_plus_diff X-Git-Multimail-Version: 1.5.dev Auto-Submitted: auto-generated This is an automated email from the ASF dual-hosted git repository. mwalch pushed a commit to branch asf-site in repository https://gitbox.apache.org/repos/asf/accumulo-website.git The following commit(s) were added to refs/heads/asf-site by this push: new 8ae53ce Jekyll build from master:2bab009 8ae53ce is described below commit 8ae53ce5bbf8347e2334f3b29d7a8b031ebc12e3 Author: Mike Walch AuthorDate: Wed Nov 28 14:15:21 2018 -0500 Jekyll build from master:2bab009 Updated MapReduce docs to include new configuration (#135) --- docs/2.x/administration/upgrading.html | 1 + docs/2.x/development/mapreduce.html | 19 +++++++++++++++++++ feed.xml | 4 ++-- search_data.json | 4 ++-- 4 files changed, 24 insertions(+), 4 deletions(-) diff --git a/docs/2.x/administration/upgrading.html b/docs/2.x/administration/upgrading.html index b4c711f..fc30597 100644 --- a/docs/2.x/administration/upgrading.html +++ b/docs/2.x/administration/upgrading.html @@ -479,6 +479,7 @@ distributions of Hadoop.
  • log4j.properties for Accumulo clients and commands
    • +
  • New Hadoop configuration is required when reading or writing to Accumulo using MapReduce.
  • Run the command accumulo shell to access the shell using configuration in conf/accumulo-client.properties
    • diff --git a/docs/2.x/development/mapreduce.html b/docs/2.x/development/mapreduce.html index d3c9e0c..76a43f0 100644 --- a/docs/2.x/development/mapreduce.html +++ b/docs/2.x/development/mapreduce.html @@ -439,6 +439,25 @@ can be set via these two format classes to do the following:

  • Restrict the input to a subset of available columns
    • +

    Configuration

    + +

    Since 2.0.0, Accumulo no longer has the same versions of dependencies (i.e Guava, etc) as Hadoop. +When launching a MapReduce job that reads or writes to Accumulo, you should build a shaded jar +with all of your dependencies and complete the following steps so YARN only includes Hadoop code +(and not all of Hadoop dependencies) when running your MapReduce job:

    + +
      +
    1. +

      Set export HADOOP_USE_CLIENT_CLASSLOADER=true in your environment before submitting +your job with yarn command.

      +
      2. +
    2. +

      Set the following in your Job configuration.

      + 
       job.getConfiguration().set("mapreduce.job.classloader", "true");
+
      +
      3. +
    +

    Mapper and Reducer classes

    To read from an Accumulo table create a Mapper with the following class diff --git a/feed.xml b/feed.xml index e166a64..41524c2 100644 --- a/feed.xml +++ b/feed.xml @@ -6,8 +6,8 @@ https://accumulo.apache.org/ - Tue, 20 Nov 2018 09:51:13 -0500 - Tue, 20 Nov 2018 09:51:13 -0500 + Wed, 28 Nov 2018 14:15:11 -0500 + Wed, 28 Nov 2018 14:15:11 -0500 Jekyll v3.7.3 diff --git a/search_data.json b/search_data.json index 31e1a86..7b96bc9 100644 --- a/search_data.json +++ b/search_data.json @@ -51,7 +51,7 @@ "docs-2-x-administration-upgrading": { "title": "Upgrading Accumulo", - "content" : "Upgrading from 1.8/9 to 2.0Follow the steps below to upgrade your Accumulo instance and client to 2.0.Upgrade Accumulo instanceIMPORTANT! Before upgrading to Accumulo 2.0, you will need to upgrade to Java 8 and Hadoop 3.x.Upgrading to Accumulo 2.0 is done by stopping Accumulo 1.8/9 and starting Accumulo 2.0.Before stopping Accumulo 1.8/9, install Accumulo 2.0 and configure it by following the 2.0 quick start.There are several changes to scripts and configuration in 2. [...] + "content" : "Upgrading from 1.8/9 to 2.0Follow the steps below to upgrade your Accumulo instance and client to 2.0.Upgrade Accumulo instanceIMPORTANT! Before upgrading to Accumulo 2.0, you will need to upgrade to Java 8 and Hadoop 3.x.Upgrading to Accumulo 2.0 is done by stopping Accumulo 1.8/9 and starting Accumulo 2.0.Before stopping Accumulo 1.8/9, install Accumulo 2.0 and configure it by following the 2.0 quick start.There are several changes to scripts and configuration in 2. [...] "url": " /docs/2.x/administration/upgrading", "categories": "administration" }, @@ -107,7 +107,7 @@ "docs-2-x-development-mapreduce": { "title": "MapReduce", - "content" : "Accumulo tables can be used as the source and destination of MapReduce jobs. Touse an Accumulo table with a MapReduce job, configure the job parameters to usethe AccumuloInputFormat and AccumuloOutputFormat. Accumulo specific parameterscan be set via these two format classes to do the following: Authenticate and provide user credentials for the input Restrict the scan to a range of rows Restrict the input to a subset of available columnsMapper and Reducer classesTo [...] + "content" : "Accumulo tables can be used as the source and destination of MapReduce jobs. Touse an Accumulo table with a MapReduce job, configure the job parameters to usethe AccumuloInputFormat and AccumuloOutputFormat. Accumulo specific parameterscan be set via these two format classes to do the following: Authenticate and provide user credentials for the input Restrict the scan to a range of rows Restrict the input to a subset of available columnsConfigurationSince 2.0.0, Ac [...] "url": " /docs/2.x/development/mapreduce", "categories": "development" },