apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Munagala Ramanath <...@datatorrent.com>
Subject Re: Serialization in Apex
Date Tue, 17 May 2016 13:06:21 GMT
Traditionally, we've recommended using
"@DefaultSerializer(JavaSerializer.class)" or
"@FieldSerializer.Bind(CustomSerializer.class)" as outlined at
http://docs.datatorrent.com/troubleshooting/#application-throwing-following-kryo-exception

Can you describe why those approaches are not adequate ?

Ram

On Mon, May 16, 2016 at 11:46 PM, Bhupesh Chawda <bhupesh@datatorrent.com>
wrote:

> Hi All,
>
> While working on the integration of Apex with Apache Samoa, I am coming
> across some scenarios where I have to add default constructors in some
> external classes to make them Kryo serializable. Although this should be
> okay, we would like to avoid modifying external classes as far as possible.
> Some other streaming engines have taken different approaches towards
> serialization.
>
> I looked at Flink and Storm serialization mechanisms.
>
> Storm has a fall back mechanism on Java serialization. It does use Kryo for
> serialization due to performance. But, if the class is not serializable
> using Kryo, then it will try to serialize it using Java serialization. If
> even then it cannot serialize, then it throws an error. [1]
>
> Flink has its own serialization stack where it uses a serializer based on
> the type information known about the data. [2]
>
> What does the community think about the current state of serialization in
> Apex. Is there a need to explore some approaches which could avoid
> serialization issues such as the one described above? Are there any other
> approaches one could use?
>
> 1.
>
> http://storm.apache.org/releases/current/Serialization.html#java-serialization
> 2.
>
> https://cwiki.apache.org/confluence/display/FLINK/Type+System,+Type+Extraction,+Serialization
>
>
> ~Bhupesh
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message