beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From kennknowles <...@git.apache.org>
Subject [GitHub] incubator-beam pull request: Exact copy of #343 trying to work aro...
Date Thu, 19 May 2016 18:52:25 GMT
GitHub user kennknowles opened a pull request:

    https://github.com/apache/incubator-beam/pull/356

    Exact copy of #343 trying to work around Jenkins issue

    Be sure to do all of the following to help us incorporate your contribution
    quickly and easily:
    
     - [x] Make sure the PR title is formatted like:
       `[BEAM-<Jira issue #>] Description of pull request`
     - [x] Make sure tests pass via `mvn clean verify`. (Even better, enable
           Travis-CI on your fork and ensure the whole test matrix passes).
     - [x] Replace `<Jira issue #>` in the title with the actual Jira issue
           number, if there is one.
     - [x] If this contribution is large, please file an Apache
           [Individual Contributor License Agreement](https://www.apache.org/licenses/icla.txt).
    
    ---
    
    From Dan's comments on [INFRA-11878](https://issues.apache.org/jira/browse/INFRA-11878)
sometimes rebasing works. It doesn't seem to be working for #343, but I am not clear on the
parameters of the issue. Trying this in case a fresh PR gets a fresh workspace from Jenkins,
or some such.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/kennknowles/incubator-beam copy-343

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/356.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #356
    
----
commit fa8bc7cd3bea3dac0484b0af139248e1f39d0317
Author: Kenneth Knowles <klk@google.com>
Date:   2016-05-02T20:11:12Z

    Add TestFlinkPipelineRunner to FlinkRunnerRegistrar
    
    This makes the runner available for selection by integration tests.

commit 3e76a85ec23da83c5ffc734c80eee882be1c3f38
Author: Kenneth Knowles <klk@google.com>
Date:   2016-05-02T21:04:20Z

    Configure RunnableOnService tests for Flink in batch mode
    
    Today Flink batch supports only global windows. This is a situation we
    intend our build to allow, eventually via JUnit category filtering.
    
    For now all the test classes that use non-global windows are excluded
    entirely via maven configuration. In the future, it should be on a
    per-test-method basis.

commit 84e10a4b0cbc78949f073737948de00bdb8179f3
Author: Aljoscha Krettek <aljoscha.krettek@gmail.com>
Date:   2016-05-06T06:26:50Z

    Fix Dangling Flink DataSets

commit f0899a682c19231f59f539e9699087ad443734cc
Author: Aljoscha Krettek <aljoscha.krettek@gmail.com>
Date:   2016-05-06T07:38:55Z

    Add hamcrest dependency to Flink Runner
    
    Without it the RunnableOnService tests seem to not work

commit 60f75655b5788b400a81abda32410eaa784f0aad
Author: Kenneth Knowles <klk@google.com>
Date:   2016-05-06T17:54:41Z

    Remove unused threadCount from integration tests

commit 8ec74ac46c10c6f1fd37d5a6049d9cb03008c432
Author: Kenneth Knowles <klk@google.com>
Date:   2016-05-06T17:55:16Z

    Disable Flink streaming integration tests for now

commit edef0ce1dcf1405e037c23f997cb1c7980e584f0
Author: Kenneth Knowles <klk@google.com>
Date:   2016-05-06T19:49:55Z

    Special casing job exec AssertionError in TestFlinkPipelineRunner

commit eadabd58f0ffca76535bbc681f051bf4292f69d9
Author: Aljoscha Krettek <aljoscha.krettek@gmail.com>
Date:   2016-05-10T11:53:03Z

    [BEAM-270] Support Timestamps/Windows in Flink Batch
    
    With this change we always use WindowedValue<T> for the underlying Flink
    DataSets instead of just T. This allows us to support windowing as well.
    
    This changes also a lot of other stuff enabled by the above:
    
     - Use WindowedValue throughout
     - Add proper translation for Window.into()
     - Make side inputs window aware
     - Make GroupByKey and Combine transformations window aware, this
       includes support for merging windows. GroupByKey is implemented as a
       Combine with a concatenating CombineFn, for simplicity
    
    This removes Flink specific transformations for things that are handled
    by builtin sources/sinks, among other things this:
    
     - Removes special translation for AvroIO.Read/Write and
       TextIO.Read/Write
     - Removes special support for Write.Bound, this was not working properly
       and is now handled by the Beam machinery that uses DoFns for this
     - Removes special translation for binary Co-Group, the code was still
       in there but was never used
    
    With this change all RunnableOnService tests run on Flink Batch.

commit 19d4e564dba1a595c164393dc525496485632e28
Author: Aljoscha Krettek <aljoscha.krettek@gmail.com>
Date:   2016-05-13T12:17:50Z

    Fix faulty Flink Flatten when PCollectionList is empty

commit 74daea464d1b4ea08660f82fdbec3c041f4e728c
Author: Aljoscha Krettek <aljoscha.krettek@gmail.com>
Date:   2016-05-13T12:41:20Z

    Remove superfluous Flink Tests, Fix those that stay in
    
    All of the stuff in the removed ITCases is covered (in more detail) by
    the RunnableOnService tests.

commit 991b2a4a75e8d5b729be9b012dc7f4ec782a9854
Author: Aljoscha Krettek <aljoscha.krettek@gmail.com>
Date:   2016-05-14T09:48:47Z

    Fix last last outstanding test

commit b5c6b1c8f7408c7f7e174df77e71ca062811800c
Author: Aljoscha Krettek <aljoscha.krettek@gmail.com>
Date:   2016-05-18T15:43:36Z

    Remove Flink-Specific ConsoleIO
    
    this can be done using a Sysout in a ParDo

commit aa179ad92bb04bf7732a6c4675b65533417ecab4
Author: Aljoscha Krettek <aljoscha.krettek@gmail.com>
Date:   2016-05-18T16:33:07Z

    Fix checkstyle warning (to force Jenkins to rerun)

commit 6c4af76fcc70a8f36acf3645ef7e0fd85d04fefe
Author: Aljoscha Krettek <aljoscha.krettek@gmail.com>
Date:   2016-05-19T12:05:05Z

    Add surefire plugin to java 8 example tests

----


---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.
---

Mime
View raw message