beam-commits mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Aviem Zur (JIRA)" <>
Subject [jira] [Commented] (BEAM-626) AvroCoder not deserializing correctly in Kryo
Date Mon, 12 Sep 2016 18:06:20 GMT


Aviem Zur commented on BEAM-626:

I believed the place to make this fix is AvroCoder since it is the only class in beam to use
writeReplace and readResolve. It is the only coder we have problems with in Kryo (using Spark
runner), while other coders, such as ProtoCoder serialize correctly and lazily initializes
its transient members. Which is why I thought the solution should be in AvroCoder rather than
manipulating Kryo, as this problem can arise in other runners.

> AvroCoder not deserializing correctly in Kryo
> ---------------------------------------------
>                 Key: BEAM-626
>                 URL:
>             Project: Beam
>          Issue Type: Bug
>          Components: sdk-java-core
>            Reporter: Aviem Zur
>            Assignee: Davor Bonaci
>            Priority: Minor
> Unlike with Java serialization, when deserializing AvroCoder using Kryo, the resulting
AvroCoder is missing all of its transient fields.
> The reason it works with Java serialization is because of the usage of writeReplace and
readResolve, which Kryo does not adhere to.
> In ProtoCoder for example there are also unserializable members, the way it is solved
there is lazy initializing these members via their getters, so they are initialized in the
deserialized object on first call to the member.
> It seems AvroCoder is the only class in Beam to use writeReplace convention.

This message was sent by Atlassian JIRA

View raw message