flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephan Ewen (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-1325) Add a closure cleaner for Java
Date Mon, 15 Dec 2014 10:42:14 GMT
Stephan Ewen created FLINK-1325:
-----------------------------------

             Summary: Add a closure cleaner for Java
                 Key: FLINK-1325
                 URL: https://issues.apache.org/jira/browse/FLINK-1325
             Project: Flink
          Issue Type: Improvement
          Components: Java API
            Reporter: Stephan Ewen


The Java API could really need a simple closure cleaner.

All functions that are implemented as anonymous subclasses hold a reference to the enclosing
class, unless they are implemented as part of a static method.

That reference (called {{this$0}}) causes serialization to fail, as it draws non serializable
classes into the function, even in cases where the function makes no access to the enclosing
data.

It is possible to manually set this reference to {{null}}, using reflection, or using a debugger.
Then the serialization succeeds.

I suggest to add a closure cleaner that uses an ASM visitor over the function's code to see
if there is any access to the {{this$0}} field. In case there is non, the field should be
set to {{null}}.

The problem can be reproduced with the simple program below:

{code}
public class Test {

  public void runProgram() throws Exception {
    ExecutionEnvironment env = ExecutionEnvironment .getExecutionEnvironment();

    env.generateSequence(1, 10)
        .map(new MapFunction<Long, Long>() {
          public Long map(Long value) {
            return value * 2;
          }
        })
        .print();

    env.execute();
  }

  public static void main(String[] args) throws Exception {
    new Test().runProgram();
  }
}
{code}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message