flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-1325) Add a closure cleaner for Java
Date Wed, 17 Dec 2014 12:41:13 GMT

    [ https://issues.apache.org/jira/browse/FLINK-1325?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14249791#comment-14249791
] 

ASF GitHub Bot commented on FLINK-1325:
---------------------------------------

GitHub user mbalassi opened a pull request:

    https://github.com/apache/incubator-flink/pull/273

    [FLINK-1325] [streaming] Added clousure cleaning to streaming

    Streaming update for @aljoscha's recent [PR](https://github.com/apache/incubator-flink/pull/269).
    
    The functionality is there, the code could be a bit nicer but as @gyfora has already refactored
the builder methods in the DataStream for the streaming [scala-api](https://github.com/mbalassi/incubator-flink/tree/scala-api)
and not yet merged it this solution will be more straightforward to merge.
    
    As this is part of the recently merged PR @aljoscha and myself are personally for adding
it to the 0.8 release (with this codebase, not the merged one). 

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/mbalassi/incubator-flink closclean

Alternatively you can review and apply these changes as the patch at:

    https://github.com/apache/incubator-flink/pull/273.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #273
    
----
commit 8d817eb9db3bdae840a7976e8d1c3c08fca6d389
Author: mbalassi <mbalassi@apache.org>
Date:   2014-12-17T12:04:30Z

    [FLINK-1325] [streaming] Added clousure cleaning to streaming

----


> Add a closure cleaner for Java
> ------------------------------
>
>                 Key: FLINK-1325
>                 URL: https://issues.apache.org/jira/browse/FLINK-1325
>             Project: Flink
>          Issue Type: Improvement
>          Components: Java API
>            Reporter: Stephan Ewen
>            Assignee: Aljoscha Krettek
>             Fix For: 0.8-incubating
>
>
> The Java API could really need a simple closure cleaner.
> All functions that are implemented as anonymous subclasses hold a reference to the enclosing
class, unless they are implemented as part of a static method.
> That reference (called {{this$0}}) causes serialization to fail, as it draws non serializable
classes into the function, even in cases where the function makes no access to the enclosing
data.
> It is possible to manually set this reference to {{null}}, using reflection, or using
a debugger. Then the serialization succeeds.
> I suggest to add a closure cleaner that uses an ASM visitor over the function's code
to see if there is any access to the {{this$0}} field. In case there is non, the field should
be set to {{null}}.
> The problem can be reproduced with the simple program below:
> {code}
> public class Test {
>   public void runProgram() throws Exception {
>     ExecutionEnvironment env = ExecutionEnvironment .getExecutionEnvironment();
>     env.generateSequence(1, 10)
>         .map(new MapFunction<Long, Long>() {
>           public Long map(Long value) {
>             return value * 2;
>           }
>         })
>         .print();
>     env.execute();
>   }
>   public static void main(String[] args) throws Exception {
>     new Test().runProgram();
>   }
> }
> {code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message