Return-Path: X-Original-To: apmail-tez-commits-archive@minotaur.apache.org Delivered-To: apmail-tez-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C357E19089 for ; Fri, 8 Apr 2016 17:16:07 +0000 (UTC) Received: (qmail 55995 invoked by uid 500); 8 Apr 2016 17:16:07 -0000 Delivered-To: apmail-tez-commits-archive@tez.apache.org Received: (qmail 55957 invoked by uid 500); 8 Apr 2016 17:16:07 -0000 Mailing-List: contact commits-help@tez.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@tez.apache.org Delivered-To: mailing list commits@tez.apache.org Received: (qmail 55948 invoked by uid 99); 8 Apr 2016 17:16:07 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 08 Apr 2016 17:16:07 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 90FDDDFC74; Fri, 8 Apr 2016 17:16:07 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: sseth@apache.org To: commits@tez.apache.org Message-Id: <8d0551549ae7416791ba16b1c6e5ed8a@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: tez git commit: TEZ-3180. Update master docs to declare hadoop-2.6.x as a minimum requirement. Contributed by Hitesh Shah. Date: Fri, 8 Apr 2016 17:16:07 +0000 (UTC) Repository: tez Updated Branches: refs/heads/master 52b732a77 -> 580d9de1c TEZ-3180. Update master docs to declare hadoop-2.6.x as a minimum requirement. Contributed by Hitesh Shah. Project: http://git-wip-us.apache.org/repos/asf/tez/repo Commit: http://git-wip-us.apache.org/repos/asf/tez/commit/580d9de1 Tree: http://git-wip-us.apache.org/repos/asf/tez/tree/580d9de1 Diff: http://git-wip-us.apache.org/repos/asf/tez/diff/580d9de1 Branch: refs/heads/master Commit: 580d9de1c8411a26517a8b9b8605cf96b22ba7e0 Parents: 52b732a Author: Siddharth Seth Authored: Fri Apr 8 10:15:46 2016 -0700 Committer: Siddharth Seth Committed: Fri Apr 8 10:15:46 2016 -0700 ---------------------------------------------------------------------- BUILDING.txt | 33 +++++---------------------------- CHANGES.txt | 1 + docs/src/site/markdown/install.md | 5 +++-- 3 files changed, 9 insertions(+), 30 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/tez/blob/580d9de1/BUILDING.txt ---------------------------------------------------------------------- diff --git a/BUILDING.txt b/BUILDING.txt index 14fd9a5..ccf8d90 100644 --- a/BUILDING.txt +++ b/BUILDING.txt @@ -11,6 +11,7 @@ Requirements: * Findbugs 2.0.2 or later (if running findbugs) * ProtocolBuffer 2.5.0 * Internet connection for first build (to fetch all dependencies) +* Hadoop version should be 2.6.0 or higher. ---------------------------------------------------------------------------------- Maven main modules: @@ -63,9 +64,9 @@ Tests options: ---------------------------------------------------------------------------------- Building against a specific version of hadoop: -Tez runs on top of Apache Hadoop YARN and requires hadoop version 2.2.0 or higher. +Tez runs on top of Apache Hadoop YARN and requires hadoop version 2.6.0 or higher. -By default, it can be compiled against hadoop versions 2.6.0 and higher by just +By default, it can be compiled against other compatible hadoop versions by just specifying the hadoop.version. For example, to build tez against hadoop 3.0.0-SNAPSHOT $ mvn package -Dhadoop.version=3.0.0-SNAPSHOT @@ -81,38 +82,14 @@ For Hadoop version X where X >= 2.8.0 $ mvn package -Dhadoop.version=${X} -Phadoop28 -P\!hadoop26 -For Hadoop version X where 2.4.0 <= X < 2.6.0 - - $ mvn package -Dhadoop.version=${X} -Phadoop24 -P\!hadoop26 - -For Hadoop version X where X < 2.4.0 - - $ mvn package -Dhadoop.version=${X} -P\!hadoop24 -P\!hadoop26 - -For recent versions of Hadoop, you can bundle AWS-S3 (2.6.0+) or Azure (2.7.0+) support +For recent versions of Hadoop (which do not bundle aws and azure by default), +you can bundle AWS-S3 (2.6.0+) or Azure (2.7.0+) support: $ mvn package -Dhadoop.version=${X} -Paws -Pazure Tez also has some shims to provide version-specific implementations for various APIs. For more details, please refer to https://cwiki.apache.org/confluence/display/TEZ/HadoopShims -For building on Windows with the hadoop24 profile enabled: - -Tez has a couple of symlinks in its codebase. Windows is a bit difficult with -respect to handling symlinks hence a couple of steps may be needed to be -able to compile the code. The symlink directories are: - -tez-plugins/tez-yarn-timeline-history/src/main/java/org/apache/tez/dag/history/logging/ats -tez-plugins/tez-yarn-timeline-history/src/test/java/org/apache/tez/tests - -To handle these symlinks, wipe these files away and copy the destination dirs -into the same location: - - - Replace tez-plugins/tez-yarn-timeline-history/src/main/java/org/apache/tez/dag/history/logging/ats -with tez-plugins/tez-yarn-timeline-history-with-acls/src/main/java/org/apache/tez/dag/history/logging/ats - - Replace tez-plugins/tez-yarn-timeline-history/src/test/java/org/apache/tez/tests -with tez-plugins/tez-yarn-timeline-history-with-acls/src/test/java/org/apache/tez/tests - ---------------------------------------------------------------------------------- UI build issues: http://git-wip-us.apache.org/repos/asf/tez/blob/580d9de1/CHANGES.txt ---------------------------------------------------------------------- diff --git a/CHANGES.txt b/CHANGES.txt index d034c8c..4ae94fe 100644 --- a/CHANGES.txt +++ b/CHANGES.txt @@ -4,6 +4,7 @@ Apache Tez Change Log Release 0.8.3: Unreleased INCOMPATIBLE CHANGES + TEZ-3180. Update master docs to declare hadoop-2.6.x as a minimum requirement. TEZ-3029. Add an onError method to service plugin contexts. TEZ-3120. Remove TaskCommContext.getCurrentDagName, Identifier. TEZ-3183. Change the taskFailed method on plugin contexts to specify the type of failure. http://git-wip-us.apache.org/repos/asf/tez/blob/580d9de1/docs/src/site/markdown/install.md ---------------------------------------------------------------------- diff --git a/docs/src/site/markdown/install.md b/docs/src/site/markdown/install.md index 3854781..a09f4bd 100644 --- a/docs/src/site/markdown/install.md +++ b/docs/src/site/markdown/install.md @@ -19,9 +19,10 @@ Install/Deploy Instructions for Tez --------------------------------------------------------------------------- -Replace x.y.z with the tez release number that you are using. E.g. 0.5.0 +Replace x.y.z with the tez release number that you are using. E.g. 0.5.0. For Tez +versions 0.8.3 and higher, Tez needs Hadoop to be of version 2.6.0 or higher. -1. Deploy Apache Hadoop using either the 2.2.0 release or a compatible 2.x version. +1. Deploy Apache Hadoop using version of 2.6.0 or higher. - You need to change the value of the hadoop.version property in the top-level pom.xml to match the version of the hadoop branch being used. ```