beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-479) Move RunnableOnService test executions to postcommit
Date Fri, 22 Jul 2016 04:17:21 GMT

    [ https://issues.apache.org/jira/browse/BEAM-479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15388880#comment-15388880
] 

ASF GitHub Bot commented on BEAM-479:
-------------------------------------

GitHub user kennknowles opened a pull request:

    https://github.com/apache/incubator-beam/pull/709

    [BEAM-479] Execute RunnableOnService tests only when runner options provided

    Be sure to do all of the following to help us incorporate your contribution
    quickly and easily:
    
     - [x] Make sure the PR title is formatted like:
       `[BEAM-<Jira issue #>] Description of pull request`
     - [ ] Make sure tests pass via `mvn clean verify`. (Even better, enable
           Travis-CI on your fork and ensure the whole test matrix passes).
     - [x] Replace `<Jira issue #>` in the title with the actual Jira issue
           number, if there is one.
     - [x] If this contribution is large, please file an Apache
           [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt).
    
    ---
    
    Previously, the situation was this:
    
     - All runners inherit a RunnableOnService integration-test
       execution referencing runnableOnServicePipelineOptions
       whether or not the variable was set. Basically an unbound
       variable reference.
     - The Dataflow runner had a profile disabling it if
       runnableOnServicePipelineOptions was not set.
     - Before they got configured, Flink and Spark had to
       do extra work to explicitly prevent the invalid
       configuration from being used.
    
    After this change:
    
     - All runners inherit the same integration-test execution
       but only if the variable it requires is present.
     - Dataflow doesn't have any special profile.
     - Flink and Spark are unchanged, since they do set
       up the variable themselves. When they move to running
       only as postcommit, like Dataflow does, the hardcoding
       is expected to either move to a profile or move to
       the Jenkins invocation.
    
    This addresses the particular aspect of [BEAM-479](https://issues.apache.org/jira/browse/BEAM-479)
about getting a symmetrical config. The way that the configuration is set up is an annoying
barrier to new runners (they have to suppress the thing) and also less readable than a straightforward
profile.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/kennknowles/incubator-beam integration-tests

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/709.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #709
    
----
commit a34a9a819d12ad4d9c14a7292fc241a89f3f836b
Author: Kenneth Knowles <klk@google.com>
Date:   2016-07-22T04:01:36Z

    Execute RunnableOnService tests only when runner options provided
    
    Previously, the situation was this:
    
     - All runners inherit a RunnableOnService integration-test
       execution referencing runnableOnServicePipelineOptions
       whether or not the variable was set. Basically an unbound
       variable reference.
     - The Dataflow runner had a profile disabling it if
       runnableOnServicePipelineOptions was not set.
     - Before they got configured, Flink and Spark had to
       do extra work to explicitly prevent the invalid
       configuration from being used.
    
    After this change:
    
     - All runners inherit the same integration-test execution
       but only if the variable it requires is present.
     - Dataflow doesn't have any special profile.
     - Flink and Spark are unchanged, since they do set
       up the variable themselves. When they move to running
       only as postcommit, like Dataflow does, the hardcoding
       is expected to either move to a profile or move to
       the Jenkins invocation.

----


> Move RunnableOnService test executions to postcommit
> ----------------------------------------------------
>
>                 Key: BEAM-479
>                 URL: https://issues.apache.org/jira/browse/BEAM-479
>             Project: Beam
>          Issue Type: Improvement
>          Components: runner-dataflow, runner-flink, runner-gearpump, runner-spark
>            Reporter: Kenneth Knowles
>            Assignee: Kenneth Knowles
>
> The Spark and Flink RunnableOnService tests only use a local endpoint, so they _can_
run as unit tests, but our test duration is getting out of hand. For Gearpump, the tests timeout.
So this ticket tracks getting everyone to a symmetric configuration.
> Later, we can re-enable a select few local endpoint tests for the various runners, to
smoke test, and hopefully get actual cluster-based integration tests running on Jenkins.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message