flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephan Ewen (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-5964) Change TypeSerializers to allow construction of immutable types
Date Mon, 06 Mar 2017 13:56:32 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15897342#comment-15897342

Stephan Ewen commented on FLINK-5964:

[~jayson.minard] Currently, Flink supports two different modes: Object *reusing* and *non-reusing*.

The *non-reusing* mode is the default and should never try to create an object just like that,
but always as a result of a copy or deserialization. The *reusing* mode can only work with
mutable objects (by definition).

Can you explain a bit more where the clash with the immutable types happens?

> Change TypeSerializers to allow construction of immutable types
> ---------------------------------------------------------------
>                 Key: FLINK-5964
>                 URL: https://issues.apache.org/jira/browse/FLINK-5964
>             Project: Flink
>          Issue Type: Improvement
>          Components: Core
>    Affects Versions: 2.0.0, 1.1.4
>            Reporter: Jayson Minard
>            Priority: Minor
> If your programming language has a lot of Immutable types (and with no default constructor)
Flink forces you to create new versions as read/write POJO otherwise the types are rejected
by the system.  In Kotlin for example, given a class and property values we can determine
which constructor to call and invoke it using knowledge of default values, nullable types
and which properties can be set in construction or after construction.
> Flink provides no opportunity to use this model because Serializers are littered with
calls to `createInstance` that are not passed any values so have no opportunity to fully inflate
the object on construction.  
> This means that when you use Flink you throw away maybe hundreds of state objects (my
worst case) and have to create Flink-only variations which becomes grunt work that adds no
> Currently TypeExtractor isn't extendable, and all of the special cases are hard coded.
 It should be configured the order of checking for type information so that pluggable types
can be added into the chain of analysis.  For example before `analyzePojo` is called I could
special case a Kotlin class returning a different TypeInformation instance.  But I don't think
that solves the whole problem since other TypeSerializers make assumptions and call `createInstance`
on other TypeSerializers without knowing how they would want to do the construction (in the
Kotlin case it would be "tell me to construct my instance and give me the list of named fields
and serializers to get their values and let me decide what to do).
> What is the best idea for this change?  With guidance, I can work on the PR.

This message was sent by Atlassian JIRA

View raw message