beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Daniel Halperin (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (BEAM-2259) Reshuffle may set watermark holds past the end of time
Date Thu, 11 May 2017 15:57:04 GMT

     [ https://issues.apache.org/jira/browse/BEAM-2259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Daniel Halperin updated BEAM-2259:
----------------------------------
    Fix Version/s: 2.0.0

> Reshuffle may set watermark holds past the end of time
> ------------------------------------------------------
>
>                 Key: BEAM-2259
>                 URL: https://issues.apache.org/jira/browse/BEAM-2259
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-core
>            Reporter: Thomas Groh
>            Assignee: Thomas Groh
>             Fix For: 2.0.0
>
>
> When it does so, the ReduceFnRunner throws an exception that the watermark hold is past
the end of time
> This affects all non-Dataflow runners.
> (With additional checks)
> Caused by: java.lang.IllegalArgumentException: Assigned TimestampedValueInSingleWindow{value=KV{key:
gameexamples:test_tgroh.l
> eaderboard_team shard: 33, org.apache.beam.sdk.io.gcp.bigquery.TableRowInfo@31436a52},
timestamp=2017-05-11T15:29:59.999Z, window=[20
> 17-05-11T15:15:00.000Z..2017-05-11T15:30:00.000Z), pane=PaneInfo{isFirst=true, timing=EARLY,
index=0}} to a window [2017-05-11T15:15:
> 00.000Z..2017-05-11T15:30:00.000Z) with fn org.apache.beam.sdk.util.IdentityWindowFn@58543f9f.
Expiry is past end of time. Windowing 
> strategy WindowingStrategy{windowFn=org.apache.beam.sdk.util.IdentityWindowFn@58543f9f,
allowedLateness=PT9223372036854.775S, trigger
> =ReshuffleTrigger(), accumulationMode=DISCARDING_FIRED_PANES, timestampCombiner=EARLIEST}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message