flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: Immutable data
Date Wed, 23 Sep 2015 12:02:15 GMT
Hi Jack,
Stephan is right, this should work. Unfortunately the TypeAnalyzer does not
correctly detect that it cannot treat your Id class as a Pojo. I will add a
Jira issue for that. For the time being you can use this command to force
the system to use Kryo:

env.getConfig.enableForceKryo();

I hope this helps.

Regards,
Aljoscha


On Wed, 23 Sep 2015 at 13:37 Stephan Ewen <sewen@apache.org> wrote:

> Hi Jack!
>
> This should be supported, there is no strict requirement for mutable types.
>
> The POJO rules apply only if you want to use the "by-field-name"
> addressing for keys. In Scala, you should be able to use case classes as
> well, even if they are immutable.
>
> Can you post the exception that you get?
>
> Greetings,
> Stephan
>
>
> On Wed, Sep 23, 2015 at 1:29 PM, Jack <jack-knilf@marmelandia.com> wrote:
>
>> Hi,
>>
>> I'm having trouble integrating existing Scala code with Flink, due to
>> POJO-only requirement.
>>
>> We're using AnyVal heavily for type safety, and immutable classes as a
>> default. For example, the following does not work:
>>
>> object Test {
>>   class Id(val underlying: Int) extends AnyVal
>>
>>   class X(var id: Id) {
>>     def this() { this(new Id(0)) }
>>   }
>>
>>   class MySource extends SourceFunction[X] {
>>     def run(ctx: SourceFunction.SourceContext[X]) {
>>       ctx.collect(new X(new Id(1)))
>>     }
>>     def cancel() {}
>>   }
>>
>>   def main(args: Array[String]) {
>>     val env = StreamExecutionContext.getExecutionContext
>>     env.addSource(new MySource).print
>>     env.execute("Test")
>>   }
>> }
>>
>> Currently I'm thinking that I would need to have duplicate classes and
>> code for Flint and for non-Flint code, or somehow use immutable interfaces
>> for non-Flint code. Both ways are expensive in terms of development time.
>>
>> Would you have any guidance on how to integrate Flink with a code base
>> that has immutability as a norm?
>>
>> Thanks
>
>
>

Mime
View raw message