spark-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From pwend...@apache.org
Subject [24/50] git commit: Minor doc fixes and updating README
Date Thu, 12 Dec 2013 07:11:55 GMT
Minor doc fixes and updating README


Project: http://git-wip-us.apache.org/repos/asf/incubator-spark/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-spark/commit/bb6e25c6
Tree: http://git-wip-us.apache.org/repos/asf/incubator-spark/tree/bb6e25c6
Diff: http://git-wip-us.apache.org/repos/asf/incubator-spark/diff/bb6e25c6

Branch: refs/heads/scala-2.10
Commit: bb6e25c663a0fa96552994bcdda2049e9b621db7
Parents: 241336a
Author: Patrick Wendell <pwendell@gmail.com>
Authored: Fri Dec 6 17:41:27 2013 -0800
Committer: Patrick Wendell <pwendell@gmail.com>
Committed: Fri Dec 6 17:42:28 2013 -0800

----------------------------------------------------------------------
 README.md                   | 7 ++++++-
 docs/building-with-maven.md | 4 +++-
 docs/index.md               | 2 +-
 3 files changed, 10 insertions(+), 3 deletions(-)
----------------------------------------------------------------------


http://git-wip-us.apache.org/repos/asf/incubator-spark/blob/bb6e25c6/README.md
----------------------------------------------------------------------
diff --git a/README.md b/README.md
index 456b806..8c7853e 100644
--- a/README.md
+++ b/README.md
@@ -55,7 +55,7 @@ versions without YARN, use:
     # Cloudera CDH 4.2.0 with MapReduce v1
     $ SPARK_HADOOP_VERSION=2.0.0-mr1-cdh4.2.0 sbt/sbt assembly
 
-For Apache Hadoop 2.x, 0.23.x, Cloudera CDH MRv2, and other Hadoop versions
+For Apache Hadoop 2.0.X, 2.1.X, 0.23.x, Cloudera CDH MRv2, and other Hadoop versions
 with YARN, also set `SPARK_YARN=true`:
 
     # Apache Hadoop 2.0.5-alpha
@@ -64,6 +64,11 @@ with YARN, also set `SPARK_YARN=true`:
     # Cloudera CDH 4.2.0 with MapReduce v2
     $ SPARK_HADOOP_VERSION=2.0.0-cdh4.2.0 SPARK_YARN=true sbt/sbt assembly
 
+When building for Hadoop 2.2.X and newer, you'll need to include the additional `new-yarn`
profile:
+
+    # Apache Hadoop 2.2.X and newer
+    $ mvn -Dyarn.version=2.2.0 -Dhadoop.version=2.2.0 -Pnew-yarn
+
 For convenience, these variables may also be set through the `conf/spark-env.sh` file
 described below.
 

http://git-wip-us.apache.org/repos/asf/incubator-spark/blob/bb6e25c6/docs/building-with-maven.md
----------------------------------------------------------------------
diff --git a/docs/building-with-maven.md b/docs/building-with-maven.md
index a508786..c709001 100644
--- a/docs/building-with-maven.md
+++ b/docs/building-with-maven.md
@@ -46,7 +46,9 @@ For Apache Hadoop 2.x, 0.23.x, Cloudera CDH MRv2, and other Hadoop versions
with
     $ mvn -Phadoop2-yarn -Dhadoop.version=2.0.0-cdh4.2.0 -Dyarn.version=2.0.0-chd4.2.0 -DskipTests
clean package
 
 Hadoop versions 2.2.x and newer can be built by setting the ```new-yarn``` and the ```yarn.version```
as follows:
-       mvn -Dyarn.version=2.2.0 -Dhadoop.version=2.2.0 -Pnew-yarn
+ 
+    # Apache Hadoop 2.2.X and newer
+    $ mvn -Dyarn.version=2.2.0 -Dhadoop.version=2.2.0 -Pnew-yarn
 
 The build process handles Hadoop 2.2.x as a special case that uses the directory ```new-yarn```,
which supports the new YARN API. Furthermore, for this version, the build depends on artifacts
published by the spark-project to enable Akka 2.0.5 to work with protobuf 2.5. 
 

http://git-wip-us.apache.org/repos/asf/incubator-spark/blob/bb6e25c6/docs/index.md
----------------------------------------------------------------------
diff --git a/docs/index.md b/docs/index.md
index bbb2733..45616f7 100644
--- a/docs/index.md
+++ b/docs/index.md
@@ -65,7 +65,7 @@ In addition, if you wish to run Spark on [YARN](running-on-yarn.md), set
 
 Note that on Windows, you need to set the environment variables on separate lines, e.g.,
`set SPARK_HADOOP_VERSION=1.2.1`.
 
-For this version of Spark (0.8.1) Hadoop 2.2.x (or newer) users will have to build Spark
and publish it locally. See [Launching Spark on YARN](running-on-yarn.md). This is needed
because Hadoop 2.2 has non backwards compatible API changes.
+For this version of Spark (0.8.1) Hadoop 2.2.x (or newer) users will have to build Spark
and publish it locally. See [Launching Spark on YARN](running-on-yarn.html). This is needed
because Hadoop 2.2 has non backwards compatible API changes.
 
 # Where to Go from Here
 


Mime
View raw message