spark-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Graham Dennis (JIRA)" <>
Subject [jira] [Updated] (SPARK-2878) Inconsistent Kryo serialisation with custom Kryo Registrator
Date Wed, 06 Aug 2014 09:59:12 GMT


Graham Dennis updated SPARK-2878:

    Affects Version/s: 1.0.2

> Inconsistent Kryo serialisation with custom Kryo Registrator
> ------------------------------------------------------------
>                 Key: SPARK-2878
>                 URL:
>             Project: Spark
>          Issue Type: Bug
>          Components: Spark Core
>    Affects Versions: 1.0.0, 1.0.2
>         Environment: Linux RedHat EL 6, 4-node Spark cluster.
>            Reporter: Graham Dennis
> The custom Kryo Registrator (a class with the org.apache.spark.serializer.KryoRegistrator
trait) is not used with every Kryo instance created, and this causes inconsistent serialisation
and deserialisation.
> The Kryo Registrator is sometimes not used because of a ClassNotFound exception that
only occurs if it *isn't* the Worker thread (of an Executor) that tries to create the KryoRegistrator.
> A complete description of the problem and a project reproducing the problem can be found
> I have currently only tested this with Spark 1.0.0, but will try to test against 1.0.2.

This message was sent by Atlassian JIRA

To unsubscribe, e-mail:
For additional commands, e-mail:

View raw message