flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yuta Morisawa <yu-moris...@kddi-research.jp>
Subject Re: Some question about document
Date Tue, 17 Jul 2018 04:27:38 GMT
Hi yang

Thank you for your comment.
I read document and got an idea.

Flink uses custom serializers on known types, and the fallback is kryo.
The other arbitary objects is serialized by kryo.



On 2018/07/12 12:14, vino yang wrote:
> Hi Yuta,
> 
> It seems Chesnay is right. The "fallback" in flink's documentation is in 
> terms of the types flink supported. But for all the other arbitrary 
> types kryo is the first choice.
> 
> 2018-07-12 9:55 GMT+08:00 Yuta Morisawa <yu-morisawa@kddi-research.jp 
> <mailto:yu-morisawa@kddi-research.jp>>:
> 
>     Thank you for your answer.
> 
>     > For POJOs Flink has a custom serializer. For arbitrary objects we use
>     > kryo, and can use Avro as a fallback.
>     https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/types_serialization.html#serialization-of-pojo-types
>     <https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/types_serialization.html#serialization-of-pojo-types>
> 
>     It may be the reverse.
>     Kryo is for fallback, right?
> 
> 
>     On 2018/07/11 19:00, Chesnay Schepler wrote:
> 
>         1) TypeInformation are used to create serializers, comparators
>         and to verify correctness of certain operations (like
>         projections on tuple datasets).
> 
>         2) see
>         https://flink.apache.org/news/2015/05/11/Juggling-with-Bits-and-Bytes.html
>         <https://flink.apache.org/news/2015/05/11/Juggling-with-Bits-and-Bytes.html>
> 
>         3) Flink comes with a number of serializers for varying types as
>         outlined here
>         <https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/types_serialization.html#flinks-typeinformation-class
>         <https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/types_serialization.html#flinks-typeinformation-class>>.
>         For POJOs Flink has a custom serializer. For arbitrary objects
>         we use kryo, and can use Avro as a fallback.
> 
>         On 11.07.2018 09:24, Yuta Morisawa wrote:
> 
>             Hi all
> 
>             Now, I'm reading Flink document and I have some points to
>             feel difficult to get an idea.
>             I'd appreciate if you tell it me.
> 
>             1,TypeInformation
>               I understand TypeInformation is used for selecting
>             relevant serializer and comparator.
>               But, the ducument doesn't specify if it has another way to
>             be used.
> 
>               So, what I want to know is that what kinds of process gets
>             benefit from TypeInformation other than serializer and
>             comparator.
> 
>             2, Managed Memory
>               The word "Managed memory" is appeared several time in the
>             document but I can't find any detail description.
>               This is the only document I found
>             (https://www.slideshare.net/sbaltagi/overview-of-apacheflinkbyslimbaltagi
>             <https://www.slideshare.net/sbaltagi/overview-of-apacheflinkbyslimbaltagi>)
> 
>               If anyone has document that explains managed memory,
>             please let me know.
> 
>             3, Serializer
>               What do the words in the document  "serializers we ship
>             with Flink" mean? I know Flink uses avro for POJOs, is it
>             the same thing?
>             https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/types_serialization.html
>             <https://ci.apache.org/projects/flink/flink-docs-release-1.5/dev/types_serialization.html>
> 
> 
> 
>             Regards,
>             Yuta
> 
> 
> 
> 


Mime
View raw message