beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Commented] (BEAM-124) Testing -- End to End WordCount Batch and Streaming Tests
Date Wed, 20 Jul 2016 21:12:20 GMT


ASF GitHub Bot commented on BEAM-124:

GitHub user markflyhigh opened a pull request:

    [BEAM-124] Flink and Spark running Examples WordCountIT

    Be sure to do all of the following to help us incorporate your contribution
    quickly and easily:
     - [x] Make sure the PR title is formatted like:
       `[BEAM-<Jira issue #>] Description of pull request`
     - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
           Travis-CI on your fork and ensure the whole test matrix passes).
     - [x] Replace `<Jira issue #>` in the title with the actual Jira issue
           number, if there is one.
     - [x] If this contribution is large, please file an Apache
           [Individual Contributor License Agreement](
    +R: @dhalperi @jasonkuster
    This PR is an updated version of an old [PR](
    Removes dependencies of Spark and Flink runner on Beam java example in order to run WordCountIT
with Spark and Flink runner successfully. The following command is used for different runner:
    - Spark runner:
    `mvn clean verify -pl examples/java -am -rf :beam-examples-java -DskipITs=false -DintegrationTestPipelineOptions='[
"--tempRoot=/tmp", "--inputFile=/tmp/kinglear.txt", "--runner=org.apache.beam.runners.spark.SparkRunner"
    - Flink runner:
    `mvn clean verify -pl examples/java -am -rf :beam-examples-java -DskipITs=false -DintegrationTestPipelineOptions='[
"--tempRoot=gs://clouddfe-testing-temp-storage", "--runner=org.apache.beam.runners.flink.FlinkRunner"
    - Dataflow test runner:
    `mvn clean verify -pl examples/java -am -rf :beam-examples-java -DskipIT=false -DintegrationTestPipelineOptions='[
"--tempRoot=gs://clouddfe-testing-temp-storage", "--runner=org.apache.beam.runners.dataflow.testing.TestDataflowRunner"

You can merge this pull request into a Git repository by running:

    $ git pull wordcountit-spark-flink

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #703
commit 37753b0f3c7969ba6315c4326a958cc3c9bbbc50
Author: Mark Liu <>
Date:   2016-07-20T00:37:50Z

    [Beam-124] Run WorldCountIT with Spark and Flink runner

commit d8581c7b1bf8d14910d2f5c335caf25eba56427a
Author: Mark Liu <>
Date:   2016-07-20T18:57:11Z

    Remove depedency of Spark and Flink runner on Beam java examples


> Testing -- End to End WordCount Batch and Streaming Tests
> ---------------------------------------------------------
>                 Key: BEAM-124
>                 URL:
>             Project: Beam
>          Issue Type: New Feature
>          Components: testing
>            Reporter: Steve Wheeler
>            Assignee: Mark Liu
> Set up testing infrastructure so that an end to end test for WordCount (both batch and
streaming) will be run periodically. 

This message was sent by Atlassian JIRA

View raw message