flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stefano Baghino <stefano.bagh...@radicalbit.io>
Subject Scala/ReactiveMongo: type classes, macros and java.util.Serializable
Date Thu, 23 Jun 2016 14:22:56 GMT
Hello everybody,

in the past days, I've written batch input/output formats for MongoDB.

Initially, I've tried to use the non-blocking ReactiveMongo
<http://reactivemongo.org/> driver, which uses the type class pattern in
Scala for the serialization logic. The library also exposes some pretty
neat macros that automatically generate the type class instances for you,
given a case class.

Now, the problem is that these macros are not useful in Flink because the
generated type class instances would have to be serializable (something
that has been understandably left out from the macros). Has anyone ever
faced a similar problem? I've encountered it again when using upickle
<http://www.lihaoyi.com/upickle-pprint/upickle/>, which has a similar
facility but for JSON serialization.

In the end I've resorted to writing my own serialization logic and
explicitly extending java.util.Serializable in the end but I feel there may
be a way to not do this (without rewriting/extending the macros to make the
generated classes serializable).

Stefano Baghino

Software Engineer @ Radicalbit

View raw message