Mailing-List: contact commits-help@beam.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@beam.apache.org
Date: Thu, 6 Apr 2017 16:57:41 +0000 (UTC)
From: "Ahmet Altay (JIRA)" <jira@apache.org>
To: commits@beam.apache.org
Message-ID: <JIRA.13007725.1474926874000.226347.1491497861923@Atlassian.JIRA>
In-Reply-To: <JIRA.13007725.1474926874000@Atlassian.JIRA>
References: <JIRA.13007725.1474926874000@Atlassian.JIRA> <JIRA.13007725.1474926874567@jira-lw-us.apache.org>
Subject: [jira] [Assigned] (BEAM-681) DoFns should be serialized at apply
 time and deserialized when executing
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 06 Apr 2017 16:57:46 -0000


     [ https://issues.apache.org/jira/browse/BEAM-681?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Ahmet Altay reassigned BEAM-681:
--------------------------------

    Assignee:     (was: Frances Perry)

> DoFns should be serialized at apply time and deserialized when executing
> ------------------------------------------------------------------------
>
>                 Key: BEAM-681
>                 URL: https://issues.apache.org/jira/browse/BEAM-681
>             Project: Beam
>          Issue Type: Improvement
>          Components: sdk-py
>            Reporter: Ben Chambers
>              Labels: sdk-consistency
>
> 1. Serializing DoFns at application time ensures that any modifications of fields within the DoFn after application do not accidentally pollute the execution. This mirrors the approach taken in Java to provide an approximation of lexical-closure (eg., you only need to know the state of the DoFn at the time it was applied, not afterwards, to understand its behavior).
> 2. Based on 1, the DIrectRunner should also be deserializing DoFns before running them, which should also detect other classes of errors such as using the pipeline object (which is not pickleable) within the DoFn


--
This message was sent by Atlassian JIRA
(v6.3.15#6346)