beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kenneth Knowles (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (BEAM-32) Consider not emitting empty ON_TIME pane unless requested
Date Wed, 03 May 2017 20:46:04 GMT

    [ https://issues.apache.org/jira/browse/BEAM-32?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15995636#comment-15995636
] 

Kenneth Knowles commented on BEAM-32:
-------------------------------------

Yea, this seems like a really problematic situation.

This only happens when there was some data (to make the window active) and it was all emitted
in an early pane.

 - In accumulating mode, we will always re-emit the full result, which is fine. A user may
at some point in their pipeline filter for just the on-time result and that will work.
 - In discarding mode, it will be like a punctuation that says "you've got all the data".
In this case the user would have to filter out late data, rather than have a positive filter
for on-time. That also seems fine.

The impact of this is that users cannot safely use identity-less {{CombineFn}} in discarding
mode with early triggering. It is easy to adjoin an identity element to any type so it isn't
quite as critical as I had feared.

Noting also BEAM-210 that lets users turn it off, but I'm a bit unsure about the best default.

> Consider not emitting empty ON_TIME pane unless requested
> ---------------------------------------------------------
>
>                 Key: BEAM-32
>                 URL: https://issues.apache.org/jira/browse/BEAM-32
>             Project: Beam
>          Issue Type: New Feature
>          Components: beam-model, runner-core
>            Reporter: Kenneth Knowles
>            Priority: Minor
>              Labels: Triggers, Windowing, backward-incompatible
>             Fix For: First stable release
>
>
> Today, the ReduceFnRunner sets a timers and emits an empty ON_TIME pane as long as the
trigger allows it. This could be controlled in a manner analogous to the empty final pane
at window expiration (also owned by the ReduceFnRunner).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message