Return-Path: X-Original-To: apmail-flink-commits-archive@minotaur.apache.org Delivered-To: apmail-flink-commits-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id D8C7917367 for ; Thu, 24 Sep 2015 14:13:02 +0000 (UTC) Received: (qmail 86300 invoked by uid 500); 24 Sep 2015 14:12:59 -0000 Delivered-To: apmail-flink-commits-archive@flink.apache.org Received: (qmail 86258 invoked by uid 500); 24 Sep 2015 14:12:59 -0000 Mailing-List: contact commits-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: dev@flink.apache.org Delivered-To: mailing list commits@flink.apache.org Received: (qmail 86244 invoked by uid 99); 24 Sep 2015 14:12:59 -0000 Received: from git1-us-west.apache.org (HELO git1-us-west.apache.org) (140.211.11.23) by apache.org (qpsmtpd/0.29) with ESMTP; Thu, 24 Sep 2015 14:12:59 +0000 Received: by git1-us-west.apache.org (ASF Mail Server at git1-us-west.apache.org, from userid 33) id 6AED3E10C6; Thu, 24 Sep 2015 14:12:59 +0000 (UTC) Content-Type: text/plain; charset="us-ascii" MIME-Version: 1.0 Content-Transfer-Encoding: 7bit From: rmetzger@apache.org To: commits@flink.apache.org Date: Thu, 24 Sep 2015 14:12:59 -0000 Message-Id: <525663fd49414eb2b4b7ec794191804b@git.apache.org> X-Mailer: ASF-Git Admin Mailer Subject: [1/2] flink git commit: [FLINK-2200] [build system] Add scala version suffix to artifact id for Scala 2.11 Repository: flink Updated Branches: refs/heads/master 0d19e94a1 -> 4cadc3d60 [FLINK-2200] [build system] Add scala version suffix to artifact id for Scala 2.11 Project: http://git-wip-us.apache.org/repos/asf/flink/repo Commit: http://git-wip-us.apache.org/repos/asf/flink/commit/7f07cf8a Tree: http://git-wip-us.apache.org/repos/asf/flink/tree/7f07cf8a Diff: http://git-wip-us.apache.org/repos/asf/flink/diff/7f07cf8a Branch: refs/heads/master Commit: 7f07cf8a7d0799501925859ea8e8b02a71a77bdc Parents: 0d19e94 Author: Chiwan Park Authored: Fri Jul 3 02:28:51 2015 +0900 Committer: Robert Metzger Committed: Thu Sep 24 16:11:46 2015 +0200 ---------------------------------------------------------------------- docs/_config.yml | 3 - docs/apis/programming_guide.md | 12 ++- docs/index.md | 6 +- docs/setup/building.md | 4 +- flink-contrib/flink-operator-stats/pom.xml | 2 +- flink-dist/src/main/assemblies/bin.xml | 12 +-- .../main/resources/archetype-resources/pom.xml | 2 +- .../main/resources/archetype-resources/pom.xml | 2 +- flink-quickstart/pom.xml | 2 +- flink-runtime-web/pom.xml | 2 +- flink-staging/flink-gelly-scala/pom.xml | 2 +- tools/change-scala-version.sh | 81 ++++++++++++++++++++ 12 files changed, 109 insertions(+), 21 deletions(-) ---------------------------------------------------------------------- http://git-wip-us.apache.org/repos/asf/flink/blob/7f07cf8a/docs/_config.yml ---------------------------------------------------------------------- diff --git a/docs/_config.yml b/docs/_config.yml index 059340d..ae69328 100644 --- a/docs/_config.yml +++ b/docs/_config.yml @@ -31,9 +31,6 @@ version: "0.10-SNAPSHOT" version_hadoop1: "0.10-hadoop1-SNAPSHOT" version_short: "0.10" # Used for the top navbar w/o snapshot suffix -# This is the Scala version we use in this version -scala_version: "2.10" - # Some commonly linked pages (this was more important to have as a variable # during incubator; by now it should also be fine to hardcode these.) website_url: "http://flink.apache.org" http://git-wip-us.apache.org/repos/asf/flink/blob/7f07cf8a/docs/apis/programming_guide.md ---------------------------------------------------------------------- diff --git a/docs/apis/programming_guide.md b/docs/apis/programming_guide.md index a9c7880..ff831a9 100644 --- a/docs/apis/programming_guide.md +++ b/docs/apis/programming_guide.md @@ -136,7 +136,7 @@ mvn archetype:generate / -The archetypes are working for stable releases and preview versions (`-SNAPSHOT`) +The archetypes are working for stable releases and preview versions (`-SNAPSHOT`). If you want to add Flink to an existing Maven project, add the following entry to your *dependencies* section in the *pom.xml* file of your project: @@ -187,7 +187,17 @@ that creates the type information for Flink operations. +#### Scala Dependency Versions +Because Scala 2.10 binary is not compatible with Scala 2.11 binary, we provide multiple artifacts +to support both Scala versions. + +Starting from the 0.10 line, we cross-build all Flink modules for both 2.10 and 2.11. If you want +to run your program on Flink with Scala 2.11, you need to add a `_2.11` suffix to the `artifactId` +values of the Flink modules in your dependencies section. + +If you are looking for building Flink with Scala 2.11, please check +[build guide](../setup/building.html#build-flink-for-scala-211). #### Hadoop Dependency Versions http://git-wip-us.apache.org/repos/asf/flink/blob/7f07cf8a/docs/index.md ---------------------------------------------------------------------- diff --git a/docs/index.md b/docs/index.md index f03ac49..80ed836 100644 --- a/docs/index.md +++ b/docs/index.md @@ -64,9 +64,11 @@ You can download the latest pre-built snapshot version from the [downloads]({{ s The Scala API uses Scala {{ site.scala_version }}. Please make sure to use a compatible version. +Basically, the Scala API uses Scala 2.10. But you can use the API with Scala 2.11. To use Flink with +Scala 2.11, please check [build guide](/setup/building.html#build-flink-for-scala-211) +and [programming guide](/apis/programming_guide.html#scala-dependency-versions). + ## Flink Architecture Flink Process Model - - http://git-wip-us.apache.org/repos/asf/flink/blob/7f07cf8a/docs/setup/building.md ---------------------------------------------------------------------- diff --git a/docs/setup/building.md b/docs/setup/building.md index ce59761..2f51da6 100644 --- a/docs/setup/building.md +++ b/docs/setup/building.md @@ -96,7 +96,6 @@ So if you are building Flink for Hadoop `2.0.0-alpha`, use the following command -P!include-yarn -Dhadoop.version=2.0.0-alpha ~~~ - ## Build Flink for a specific Scala Version **Note:** Users that purely use the Java APIs and libraries can ignore this section. @@ -110,11 +109,10 @@ By default, Flink is built with Scala *2.10*. To build Flink with Scala *2.11*, mvn clean install -DskipTests -Dscala-2.11 ~~~ - To build against custom Scala versions, you need to supply the *language version* and the *binary version* as properties to the build: ~~~bash -mvn clean install -DskipTests -Dscala.version=2.11.4 -Dscala.binary.version=2.11 +mvn clean install -DskipTests -Dscala-2.11 -Dscala.version=2.11.4 -Dscala.binary.version=2.11 ~~~ Flink is developed against Scala *2.10*, and tested additionally against Scala *2.11*. These two versions are known to be compatible. Earlier versions (like Scala *2.9*) are *not* compatible. http://git-wip-us.apache.org/repos/asf/flink/blob/7f07cf8a/flink-contrib/flink-operator-stats/pom.xml ---------------------------------------------------------------------- diff --git a/flink-contrib/flink-operator-stats/pom.xml b/flink-contrib/flink-operator-stats/pom.xml index fb2f3e6..d85e1a6 100644 --- a/flink-contrib/flink-operator-stats/pom.xml +++ b/flink-contrib/flink-operator-stats/pom.xml @@ -64,4 +64,4 @@ under the License. - \ No newline at end of file + http://git-wip-us.apache.org/repos/asf/flink/blob/7f07cf8a/flink-dist/src/main/assemblies/bin.xml ---------------------------------------------------------------------- diff --git a/flink-dist/src/main/assemblies/bin.xml b/flink-dist/src/main/assemblies/bin.xml index 960fd30..270fb17 100644 --- a/flink-dist/src/main/assemblies/bin.xml +++ b/flink-dist/src/main/assemblies/bin.xml @@ -155,12 +155,12 @@ under the License. *.jar - flink-java-examples-${project.version}.jar - original-flink-java-examples-${project.version}.jar - flink-java-examples-${project.version}-sources.jar - flink-java-examples-${project.version}-tests.jar - flink-java-examples-${project.version}-javadoc.jar - flink-java-examples-${project.version}-*.jar + flink-java-examples*-${project.version}.jar + original-flink-java-examples*-${project.version}.jar + flink-java-examples*-${project.version}-sources.jar + flink-java-examples*-${project.version}-tests.jar + flink-java-examples*-${project.version}-javadoc.jar + flink-java-examples*-${project.version}-*.jar http://git-wip-us.apache.org/repos/asf/flink/blob/7f07cf8a/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/pom.xml ---------------------------------------------------------------------- diff --git a/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/pom.xml b/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/pom.xml index 49bfb76..8697e53 100644 --- a/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/pom.xml +++ b/flink-quickstart/flink-quickstart-java/src/main/resources/archetype-resources/pom.xml @@ -96,7 +96,7 @@ under the License. org.apache.maven.plugins maven-shade-plugin - 2.3 + 2.4.1 http://git-wip-us.apache.org/repos/asf/flink/blob/7f07cf8a/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/pom.xml ---------------------------------------------------------------------- diff --git a/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/pom.xml b/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/pom.xml index f01d82d..3c2345f 100644 --- a/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/pom.xml +++ b/flink-quickstart/flink-quickstart-scala/src/main/resources/archetype-resources/pom.xml @@ -100,7 +100,7 @@ under the License. org.apache.maven.plugins maven-shade-plugin - 2.3 + 2.4.1 http://git-wip-us.apache.org/repos/asf/flink/blob/7f07cf8a/flink-quickstart/pom.xml ---------------------------------------------------------------------- diff --git a/flink-quickstart/pom.xml b/flink-quickstart/pom.xml index c5f55df..0a3fcc2 100644 --- a/flink-quickstart/pom.xml +++ b/flink-quickstart/pom.xml @@ -84,4 +84,4 @@ under the License. - \ No newline at end of file + http://git-wip-us.apache.org/repos/asf/flink/blob/7f07cf8a/flink-runtime-web/pom.xml ---------------------------------------------------------------------- diff --git a/flink-runtime-web/pom.xml b/flink-runtime-web/pom.xml index 804e6da..afe71b4 100644 --- a/flink-runtime-web/pom.xml +++ b/flink-runtime-web/pom.xml @@ -90,7 +90,7 @@ under the License. guava ${guava.version} - + http://git-wip-us.apache.org/repos/asf/flink/blob/7f07cf8a/flink-staging/flink-gelly-scala/pom.xml ---------------------------------------------------------------------- diff --git a/flink-staging/flink-gelly-scala/pom.xml b/flink-staging/flink-gelly-scala/pom.xml index 390dbb8..18c0516 100644 --- a/flink-staging/flink-gelly-scala/pom.xml +++ b/flink-staging/flink-gelly-scala/pom.xml @@ -195,4 +195,4 @@ under the License. - \ No newline at end of file + http://git-wip-us.apache.org/repos/asf/flink/blob/7f07cf8a/tools/change-scala-version.sh ---------------------------------------------------------------------- diff --git a/tools/change-scala-version.sh b/tools/change-scala-version.sh new file mode 100755 index 0000000..ff38711 --- /dev/null +++ b/tools/change-scala-version.sh @@ -0,0 +1,81 @@ +#!/usr/bin/env bash + +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements. See the NOTICE file distributed with +# this work for additional information regarding copyright ownership. +# The ASF licenses this file to You under the Apache License, Version 2.0 +# (the "License"); you may not use this file except in compliance with +# the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, software +# distributed under the License is distributed on an "AS IS" BASIS, +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. +# See the License for the specific language governing permissions and +# limitations under the License. +# + +# This shell script is from Apache Spark with some modification. + +set -e + +VALID_VERSIONS=( 2.10 2.11 ) + +usage() { + echo "Usage: $(basename $0) [-h|--help] +where : + -h| --help Display this help text + valid scala version values : ${VALID_VERSIONS[*]} +" 1>&2 + exit 1 +} + +if [[ ($# -ne 1) || ( $1 == "--help") || $1 == "-h" ]]; then + usage +fi + +TO_VERSION=$1 + +check_scala_version() { + for i in ${VALID_VERSIONS[*]}; do [ $i = "$1" ] && return 0; done + echo "Invalid Scala version: $1. Valid versions: ${VALID_VERSIONS[*]}" 1>&2 + exit 1 +} + +check_scala_version "$TO_VERSION" + +if [ $TO_VERSION = "2.11" ]; then + FROM_SUFFIX="" + TO_SUFFIX="_2.11" +else + FROM_SUFFIX="_2\.11" + TO_SUFFIX="" +fi + +sed_i() { + sed -e "$1" "$2" > "$2.tmp" && mv "$2.tmp" "$2" +} + +export -f sed_i + +echo "sed_i 's/\(artifactId>flink.*'$FROM_SUFFIX'\)<\/artifactId>/\1'$TO_SUFFIX'<\/artifactId>/g' {}"; + +BASEDIR=$(dirname $0)/.. +find "$BASEDIR" -name 'pom.xml' -not -path '*target*' -print \ + -exec bash -c "sed_i 's/\(artifactId>flink.*\)'$FROM_SUFFIX'<\/artifactId>/\1'$TO_SUFFIX'<\/artifactId>/g' {}" \; + +# fix for examples +find "$BASEDIR/flink-examples/flink-java-examples" -name 'pom.xml' -not -path '*target*' -print \ + -exec bash -c "sed_i 's/\(org\.apache\.flink:flink-.*\)'$FROM_SUFFIX'<\/exclude>/\1'$TO_SUFFIX'<\/exclude>/g' {}" \; + +# fix for flink-dist +find "$BASEDIR/flink-dist" -name 'bin.xml' -not -path '*target*' -print \ + -exec bash -c "sed_i 's/\(.*flink-dist\)'$FROM_SUFFIX'/\1'$TO_SUFFIX'/g' {}" \; +find "$BASEDIR/flink-dist" -name 'bin.xml' -not -path '*target*' -print \ + -exec bash -c "sed_i 's/\(org\.apache\.flink:flink-.*\)'$FROM_SUFFIX'<\/include>/\1'$TO_SUFFIX'<\/include>/g' {}" \;