Return-Path: X-Original-To: apmail-spark-commits-archive@minotaur.apache.org Delivered-To: apmail-spark-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id C2FF7C8D9 for ; Fri, 14 Nov 2014 22:22:22 +0000 (UTC) Received: (qmail 76433 invoked by uid 500); 14 Nov 2014 22:22:22 -0000 Delivered-To: apmail-spark-commits-archive@spark.apache.org Received: (qmail 76308 invoked by uid 500); 14 Nov 2014 22:22:22 -0000 Mailing-List: contact commits-help@spark.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list commits@spark.apache.org Received: (qmail 76297 invoked by uid 99); 14 Nov 2014 22:22:22 -0000 Received: from tyr.zones.apache.org (HELO tyr.zones.apache.org) (140.211.11.114) by apache.org (qpsmtpd/0.29) with ESMTP; Fri, 14 Nov 2014 22:22:22 +0000 Received: by tyr.zones.apache.org (Postfix, from userid 65534) id 1552198A5D2; Fri, 14 Nov 2014 22:22:22 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: pwendell@apache.org To: commits@spark.apache.org Message-Id: <98509837005b4545a2f461b507f819f1@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: spark git commit: SPARK-4375. no longer require -Pscala-2.10 Date: Fri, 14 Nov 2014 22:22:22 +0000 (UTC) Repository: spark Updated Branches: refs/heads/branch-1.2 4bdeeb7d2 -> d90ddf12b SPARK-4375. no longer require -Pscala-2.10 It seems like the winds might have moved away from this approach, but wanted to post the PR anyway because I got it working and to show what it would look like. Author: Sandy Ryza Closes #3239 from sryza/sandy-spark-4375 and squashes the following commits: 0ffbe95 [Sandy Ryza] Enable -Dscala-2.11 in sbt cd42d94 [Sandy Ryza] Update doc f6644c3 [Sandy Ryza] SPARK-4375 take 2 (cherry picked from commit f5f757e4ed80759dc5668c63d5663651689f8da8) Signed-off-by: Patrick Wendell Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/d90ddf12 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/d90ddf12 Diff: http://git-wip-us.apache.org/repos/asf/spark/diff/d90ddf12 Branch: refs/heads/branch-1.2 Commit: d90ddf12b6bea2162e982e800c96d2c94f66b347 Parents: 4bdeeb7 Author: Sandy Ryza Authored: Fri Nov 14 14:21:57 2014 -0800 Committer: Patrick Wendell Committed: Fri Nov 14 14:22:13 2014 -0800 ---------------------------------------------------------------------- docs/building-spark.md | 4 +- examples/pom.xml | 65 ++--------------------- pom.xml | 22 +------- project/SparkBuild.scala | 3 ++ repl/pom.xml | 117 +++++++++++++++--------------------------- sql/catalyst/pom.xml | 11 +--- sql/hive/pom.xml | 3 -- 7 files changed, 54 insertions(+), 171 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/spark/blob/d90ddf12/docs/building-spark.md ---------------------------------------------------------------------- diff --git a/docs/building-spark.md b/docs/building-spark.md index 20ba7da..bb18414 100644 --- a/docs/building-spark.md +++ b/docs/building-spark.md @@ -113,9 +113,9 @@ mvn -Pyarn -Phive -Phive-thriftserver-0.12.0 -Phadoop-2.4 -Dhadoop.version=2.4.0 {% endhighlight %} # Building for Scala 2.11 -To produce a Spark package compiled with Scala 2.11, use the `-Pscala-2.11` profile: +To produce a Spark package compiled with Scala 2.11, use the `-Dscala-2.11` property: - mvn -Pyarn -Phadoop-2.4 -Pscala-2.11 -DskipTests clean package + mvn -Pyarn -Phadoop-2.4 -Dscala-2.11 -DskipTests clean package Scala 2.11 support in Spark is experimental and does not support a few features. Specifically, Spark's external Kafka library and JDBC component are not yet http://git-wip-us.apache.org/repos/asf/spark/blob/d90ddf12/examples/pom.xml ---------------------------------------------------------------------- diff --git a/examples/pom.xml b/examples/pom.xml index 2ec5728..2752ce3 100644 --- a/examples/pom.xml +++ b/examples/pom.xml @@ -389,11 +389,11 @@ - + scala-2.10 - true + !scala-2.11 @@ -427,65 +427,6 @@ - - add-scala-test-sources - generate-test-sources - - add-test-source - - - - src/test/scala - scala-2.10/src/test/scala - scala-2.10/src/test/java - - - - - - - - - - scala-2.11 - - false - - - - - - - - org.codehaus.mojo - build-helper-maven-plugin - - - add-scala-sources - generate-sources - - add-source - - - - src/main/scala - scala-2.11/src/main/scala - - - - - add-scala-test-sources - generate-test-sources - - add-test-source - - - - src/test/scala - scala-2.11/src/test/scala - - - http://git-wip-us.apache.org/repos/asf/spark/blob/d90ddf12/pom.xml ---------------------------------------------------------------------- diff --git a/pom.xml b/pom.xml index 7bbde31..90d3bff 100644 --- a/pom.xml +++ b/pom.xml @@ -1296,9 +1296,6 @@ mapr3 - - false - 1.0.3-mapr-3.0.3 2.3.0-mapr-4.0.0-FCS @@ -1309,9 +1306,6 @@ mapr4 - - false - 2.3.0-mapr-4.0.0-FCS 2.3.0-mapr-4.0.0-FCS @@ -1341,9 +1335,6 @@ hadoop-provided - - false - org.apache.hadoop @@ -1390,18 +1381,12 @@ hive-thriftserver - - false - sql/hive-thriftserver hive-0.12.0 - - false - 0.12.0-protobuf-2.5 0.12.0 @@ -1410,9 +1395,6 @@ hive-0.13.1 - - false - 0.13.1a 0.13.1 @@ -1423,7 +1405,7 @@ scala-2.10 - true + !scala-2.11 2.10.4 @@ -1439,7 +1421,7 @@ scala-2.11 - false + scala-2.11 2.11.2 http://git-wip-us.apache.org/repos/asf/spark/blob/d90ddf12/project/SparkBuild.scala ---------------------------------------------------------------------- diff --git a/project/SparkBuild.scala b/project/SparkBuild.scala index d95d50a..c96a6c4 100644 --- a/project/SparkBuild.scala +++ b/project/SparkBuild.scala @@ -100,8 +100,11 @@ object SparkBuild extends PomBuild { "conjunction with environment variable.") v.split("(\\s+|,)").filterNot(_.isEmpty).map(_.trim.replaceAll("-P", "")).toSeq } + if (profiles.exists(_.contains("scala-"))) { profiles + } else if (System.getProperty("scala-2.11") != null) { + profiles ++ Seq("scala-2.11") } else { println("Enabled default scala profile") profiles ++ Seq("scala-2.10") http://git-wip-us.apache.org/repos/asf/spark/blob/d90ddf12/repl/pom.xml ---------------------------------------------------------------------- diff --git a/repl/pom.xml b/repl/pom.xml index bd688c8..c2bf9fd 100644 --- a/repl/pom.xml +++ b/repl/pom.xml @@ -35,6 +35,8 @@ repl /usr/share/spark root + scala-2.10/src/main/scala + scala-2.10/src/test/scala @@ -122,86 +124,51 @@ + + + org.codehaus.mojo + build-helper-maven-plugin + + + add-scala-sources + generate-sources + + add-source + + + + src/main/scala + ${extra.source.dir} + + + + + add-scala-test-sources + generate-test-sources + + add-test-source + + + + src/test/scala + ${extra.testsource.dir} + + + + + - scala-2.10 - - - - org.codehaus.mojo - build-helper-maven-plugin - - - add-scala-sources - generate-sources - - add-source - - - - src/main/scala - scala-2.10/src/main/scala - - - - - add-scala-test-sources - generate-test-sources - - add-test-source - - - - src/test/scala - scala-2.10/src/test/scala - - - - - - - - - scala-2.11 - - - - org.codehaus.mojo - build-helper-maven-plugin - - - add-scala-sources - generate-sources - - add-source - - - - src/main/scala - scala-2.11/src/main/scala - - - - - add-scala-test-sources - generate-test-sources - - add-test-source - - - - src/test/scala - scala-2.11/src/test/scala - - - - - - - + + scala-2.11 + + + scala-2.11/src/main/scala + scala-2.11/src/test/scala + http://git-wip-us.apache.org/repos/asf/spark/blob/d90ddf12/sql/catalyst/pom.xml ---------------------------------------------------------------------- diff --git a/sql/catalyst/pom.xml b/sql/catalyst/pom.xml index 0cc3175..686d189 100644 --- a/sql/catalyst/pom.xml +++ b/sql/catalyst/pom.xml @@ -100,10 +100,11 @@ + scala-2.10 - true + !scala-2.11 @@ -113,13 +114,5 @@ - - scala-2.11 - - false - - - - http://git-wip-us.apache.org/repos/asf/spark/blob/d90ddf12/sql/hive/pom.xml ---------------------------------------------------------------------- diff --git a/sql/hive/pom.xml b/sql/hive/pom.xml index 67e36a9..fa9a1e6 100644 --- a/sql/hive/pom.xml +++ b/sql/hive/pom.xml @@ -144,9 +144,6 @@ hive-0.12.0 - - false - com.twitter --------------------------------------------------------------------- To unsubscribe, e-mail: commits-unsubscribe@spark.apache.org For additional commands, e-mail: commits-help@spark.apache.org