beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <>
Subject [jira] [Commented] (BEAM-2571) Flink ValidatesRunner failing CombineTest.testSlidingWindowsCombineWithContext
Date Thu, 20 Jul 2017 09:28:01 GMT


ASF GitHub Bot commented on BEAM-2571:

GitHub user aljoscha opened a pull request:

    [BEAM-2571] CombineTest.testSlidingWindowsCombineWithContext

    This is a rebased version of #3550. I didn't want to mess with the comments and history
on that other PR.
    R: @peihe (and the others from the older PR)

You can merge this pull request into a Git repository by running:

    $ git pull jira-2571-fix-flink-watermark-issues-rebased

Alternatively you can review and apply these changes as the patch at:

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #3600
commit 2ed36cdfef5daf124d58673933ad2e685107dd8b
Author: Alex Filatov <>
Date:   2017-07-10T10:20:49Z

    [BEAM-2306] Add checkstyle check to fail the build when @Deprecated is used without @deprecated
javadoc (or vice versa).
    The check is disabled for existing violations where reason for deprecation and/or alternative
is not clear.

commit 34dd3d616ae0c4de28e12e1f09c62cf8d26a1468
Author: Aljoscha Krettek <>
Date:   2017-07-12T12:42:37Z

    [BEAM-2571] Change DoFnOperator to use Long.MAX_VALUE as max watermark
    This is in line with what Flink does and what BoundedSourceWrapper and
    UnboundedSourceWrapper do.

commit f24ef3830c1e61379429013d712ab1e4dc65c728
Author: Pei He <>
Date:   2017-07-17T15:34:27Z

    [BEAM-2630] TestPipeline: construct job/app names based on Description in junit TestRule.

commit 9833462f9dad726391ccf6e21102eb87ad9075ec
Author: Neville Li <>
Date:   2017-07-18T13:07:21Z

    [BEAM-2532] Memoizes TableSchema in BigQuerySourceBase
    Instead of parsing the JSON schema for every record.

commit 9a8600d84b7a98ceecbe8b48ebce3b4f90976300
Author: Vikas Kedigehalli <>
Date:   2017-07-18T17:06:46Z

    Increase the gRPC message size to max value

commit 56e2c7a939ba39e4077de89e829935aea6194301
Author: Sourabh Bajaj <>
Date:   2017-07-19T17:08:14Z

    [BEAM-2636] Make sure we only override the correct class

commit 014d80ab0cdb1672359cdb2cb76a3eabf9f2fbca
Author: Luke Cwik <>
Date:   2017-07-19T18:36:35Z

    [BEAM-2642] Update Google Auth to 0.7.1
    Google auth libraries 0.6.1 and 0.7.1 have the same versions of dependencies.

commit 6cd72d72493ff68f747aef5ba69d3a850aab156c
Author: Aljoscha Krettek <>
Date:   2017-07-12T12:39:58Z

    [BEAM-2571] Clarify pushedback variable name in DoFnOperator

commit 8068ee3805ce868b7dcc28f5421bbb4782572111
Author: Aljoscha Krettek <>
Date:   2017-07-12T13:38:06Z

    [BEAM-2571] Respect watermark contract in Flink DoFnOperator
    In Flink, a watermark T specifies that there will be no elements with a
    timestamp <= T in the future. In Beam, a watermark T specifies that
    there will not be element with a timestamp < T in the future. This leads
    to problems when the watermark is exactly "on the timer timestamp", most
    prominently, this happened with Triggers, where Flink would fire the
    Trigger too early and the Trigger would determine (based on the
    watermark) that it is not yet time to fire the window while Flink
    thought it was time.
    This also adds a test that specifially tests the edge case.


> Flink ValidatesRunner failing CombineTest.testSlidingWindowsCombineWithContext
> ------------------------------------------------------------------------------
>                 Key: BEAM-2571
>                 URL:
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-flink
>            Reporter: Kenneth Knowles
>            Assignee: Aljoscha Krettek
>             Fix For: 2.2.0
> This appears to have been caused by which fixes
a couple errors in how trigger timers were processed / final panes labeled.
> I am investigating, considering roll back vs forward fix. Since it is an esoteric use
case where I would advise users to use a stateful DoFn instead, I think the bug fixed probably
outweighs the bug introduced. I would like to fix for 2.1.0 but will report back soon.

This message was sent by Atlassian JIRA

View raw message