beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-162) assert fail using session windows
Date Mon, 04 Apr 2016 22:58:25 GMT

    [ https://issues.apache.org/jira/browse/BEAM-162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15225226#comment-15225226
] 

ASF GitHub Bot commented on BEAM-162:
-------------------------------------

GitHub user mshields822 opened a pull request:

    https://github.com/apache/incubator-beam/pull/118

    [BEAM-162] Sensible interaction of merging and closed triggers

    Same bug, better approach.
     - We always check for closed triggers on the post-merged window state.
     - We are careful not to leak NEW windows.
     - More unit tests.
     - Along the way, discovered we were clearing trigger finished bits.
       Since TriggerRunner does not currently have its own unit tests,
       added new unit test in ReduceFnRunnerTest to hit that specific case.


You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/mshields822/incubator-beam beam-162

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-beam/pull/118.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #118
    
----
commit 55c34a553264491bd42db4197793eff436858b73
Author: Mark Shields <markshields@google.com>
Date:   2016-04-04T20:31:12Z

    Clear finished bits from merged-away windows, not merge result window.

commit 69eb778b354cd63fb23a48789c93ed48c2a01961
Author: Mark Shields <markshields@google.com>
Date:   2016-03-31T18:36:01Z

    Check for closed windows post-merging rather than pre-merging.
    Make sure we garbage collect NEW windows which end up being for closed windows.
    Add unit tests to confirm.

----


> assert fail using session windows
> ---------------------------------
>
>                 Key: BEAM-162
>                 URL: https://issues.apache.org/jira/browse/BEAM-162
>             Project: Beam
>          Issue Type: Bug
>          Components: runner-core
>            Reporter: Mark Shields
>            Assignee: Mark Shields
>
> java.lang.IllegalStateException: Window [2016-03-31T05:35:31.158Z..2016-03-31T06:05:31.158Z)
should have been added
> at com.google.cloud.dataflow.sdk.repackaged.com.google.common.base.Preconditions.checkState(Preconditions.java:199)
> at com.google.cloud.dataflow.sdk.util.ReduceFnRunner.processElement(ReduceFnRunner.java:440)
> at com.google.cloud.dataflow.sdk.util.ReduceFnRunner.processElements(ReduceFnRunner.java:282)
> at com.google.cloud.dataflow.sdk.util.GroupAlsoByWindowViaWindowSetDoFn.processElement(GroupAlsoByWindowViaWindowSetDoFn.java:83)
> at com.google.cloud.dataflow.sdk.util.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:49)
> at com.google.cloud.dataflow.sdk.util.DoFnRunnerBase.processElement(DoFnRunnerBase.java:138)
> at com.google.cloud.dataflow.sdk.util.LateDataDroppingDoFnRunner.processElement(LateDataDroppingDoFnRunner.java:67)
> at com.google.cloud.dataflow.sdk.runners.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:191)
> at com.google.cloud.dataflow.sdk.runners.worker.ForwardingParDoFn.processElement(ForwardingParDoFn.java:42)
> at com.google.cloud.dataflow.sdk.runners.worker.DataflowWorkerLoggingParDoFn.processElement(DataflowWorkerLoggingParDoFn.java:47)
> at com.google.cloud.dataflow.sdk.util.common.worker.ParDoOperation.process(ParDoOperation.java:53)



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message