flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Rohrmann <trohrm...@apache.org>
Subject Re: Working with protobuf wrappers
Date Tue, 01 Dec 2015 09:20:55 GMT
Hi Kryzsztof,

it's true that we once added the Protobuf serializer automatically.
However, due to versioning conflicts (see
https://issues.apache.org/jira/browse/FLINK-1635), we removed it again. Now
you have to register the ProtobufSerializer manually:
https://ci.apache.org/projects/flink/flink-docs-master/apis/best_practices.html#register-a-custom-serializer-for-your-flink-program
.

Cheers,
Till

On Mon, Nov 30, 2015 at 8:48 PM, Krzysztof Zarzycki <k.zarzycki@gmail.com>
wrote:

> Hi!
> I'm trying to use generated Protobuf wrappers compiled with protoc and
> pass them as objects between functions of Flink. I'm using Flink 0.10.0.
> Unfortunately, I get an exception on runtime:
>
> [...]
> Caused by: com.esotericsoftware.kryo.KryoException:
> java.lang.UnsupportedOperationException
> Serialization trace:
> enrichments_ (com.company$MyObject)
> at
> com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:125)
> at
> com.esotericsoftware.kryo.serializers.FieldSerializer.read(FieldSerializer.java:528)
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:657)
> at
> org.apache.flink.api.java.typeutils.runtime.kryo.KryoSerializer.copy(KryoSerializer.java:162)
> at
> org.apache.flink.streaming.runtime.tasks.OperatorChain$CopyingChainingOutput.collect(OperatorChain.java:313)
> ... 11 more
> Caused by: java.lang.UnsupportedOperationException
> at java.util.Collections$UnmodifiableCollection.add(Collections.java:1055)
> at
> com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:109)
> at
> com.esotericsoftware.kryo.serializers.CollectionSerializer.read(CollectionSerializer.java:22)
> at com.esotericsoftware.kryo.Kryo.readObject(Kryo.java:679)
> at
> com.esotericsoftware.kryo.serializers.ObjectField.read(ObjectField.java:106)
> ... 15 more
>
>
> I believed that protobuf are now serializable on default Flink
> configuration after fixing this issue in 0.9/0.8.1:
> https://issues.apache.org/jira/browse/FLINK-1392
>
> Maybe it really is, but Flink just requires some configuration?
> I'll be grateful for your help with this issue.
> Cheers,
> Krzysztof
>
>

Mime
View raw message