beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Kenneth Knowles (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (BEAM-146) WindowFn.AssingContext leaks implementation details about compressed WindowedValue representation
Date Thu, 23 Jun 2016 17:09:16 GMT

     [ https://issues.apache.org/jira/browse/BEAM-146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Kenneth Knowles resolved BEAM-146.
----------------------------------
    Resolution: Fixed

> WindowFn.AssingContext leaks implementation details about compressed WindowedValue representation
> -------------------------------------------------------------------------------------------------
>
>                 Key: BEAM-146
>                 URL: https://issues.apache.org/jira/browse/BEAM-146
>             Project: Beam
>          Issue Type: Bug
>          Components: beam-model
>            Reporter: Kenneth Knowles
>            Assignee: Kenneth Knowles
>            Priority: Minor
>
> Today, {{WindowFn.AssignContext}} provides simultaneous access to all of the windows
that a value has been placed in.
> Providing access to the current window for a value is convenient for, e.g. converting
day windows to hour windows for each hour of the assign day. But providing access to all the
assigned windows allows spooky action across windows, and is generally not intended to be
observable - elements are semantically considered to be "duplicated" into each of the assigned
windows.
> This ticket proposes that the {{AssignContext}} should provide only a single window,
and that windows should be "exploded" prior to window re-assignment so that elements are only
observed within one window at a time. This can be accomplished trivially today via surgical
insertion of {{RequiresWindowAccess}} but the {{AssignContext}} should have its API adjusted
to be explicit about it, too.
> This will affect only pipelines for which _all_ of the following hold:
>  - assigns to sliding windows (or custom {{WindowFn}} that places each element in multiple
windows)
>  - re-assigns to different windows without a {{GroupByKey}} between.
>  - the new window assignment actually does depend on the full set of windows assigned
> I hypothesize the number of such pipelines is zero.
> I expect to address this during the Beam Runner API design.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message