Mailing-List: contact commits-help@beam.incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@beam.incubator.apache.org
Date: Fri, 19 Aug 2016 19:58:20 +0000 (UTC)
From: "Daniel Halperin (JIRA)" <jira@apache.org>
To: commits@beam.incubator.apache.org
Message-ID: <JIRA.12952917.1458755374000.358099.1471636700781@Atlassian.JIRA>
In-Reply-To: <JIRA.12952917.1458755374000@Atlassian.JIRA>
References: <JIRA.12952917.1458755374000@Atlassian.JIRA> <JIRA.12952917.1458755374240@arcas>
Subject: [jira] [Updated] (BEAM-146) WindowFn.AssignContext leaks
 implementation details about compressed WindowedValue representation
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Fri, 19 Aug 2016 19:58:25 -0000


     [ https://issues.apache.org/jira/browse/BEAM-146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Daniel Halperin updated BEAM-146:
---------------------------------
    Issue Type: Improvement  (was: Bug)

> WindowFn.AssignContext leaks implementation details about compressed WindowedValue representation
> -------------------------------------------------------------------------------------------------
>
>                 Key: BEAM-146
>                 URL: https://issues.apache.org/jira/browse/BEAM-146
>             Project: Beam
>          Issue Type: Improvement
>          Components: beam-model
>            Reporter: Kenneth Knowles
>            Assignee: Kenneth Knowles
>            Priority: Minor
>             Fix For: 0.2.0-incubating
>
>
> Today, {{WindowFn.AssignContext}} provides simultaneous access to all of the windows that a value has been placed in.
> Providing access to the current window for a value is convenient for, e.g. converting day windows to hour windows for each hour of the assign day. But providing access to all the assigned windows allows spooky action across windows, and is generally not intended to be observable - elements are semantically considered to be "duplicated" into each of the assigned windows.
> This ticket proposes that the {{AssignContext}} should provide only a single window, and that windows should be "exploded" prior to window re-assignment so that elements are only observed within one window at a time. This can be accomplished trivially today via surgical insertion of {{RequiresWindowAccess}} but the {{AssignContext}} should have its API adjusted to be explicit about it, too.
> This will affect only pipelines for which _all_ of the following hold:
>  - assigns to sliding windows (or custom {{WindowFn}} that places each element in multiple windows)
>  - re-assigns to different windows without a {{GroupByKey}} between.
>  - the new window assignment actually does depend on the full set of windows assigned
> I hypothesize the number of such pipelines is zero.
> I expect to address this during the Beam Runner API design.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)