flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: Why is IdentityObjectIntMap.get called so often?
Date Tue, 24 Jan 2017 14:03:34 GMT
Hi!

I think the best way to get away from Kryo is to write types that go
through Flink's own serialization stack:
Have a look here for a bit of background:
https://ci.apache.org/projects/flink/flink-docs-release-1.2/dev/types_serialization.html#flinks-typeinformation-class

BTW: Is the "hot map" involved in Kryo's reference resolution? If yes, you
could try a custom hack and see whether it performs better when reference
resolution is deactivated (modify the KryoSerializer.checkKryoInitialized()
method to deactivate reference handling).

Hope that helps,
Stephan



On Tue, Jan 24, 2017 at 12:47 PM, Dmitry Golubets <dgolubets@gmail.com>
wrote:

> Hi,
>
> I've just added my custom MsgPack serializers hoping to see performance
> increase. I covered all data types in between chains.
>
> However this Kryo method still takes a lot of CPU: IdentityObjectIntMap.get
>
> Is there something else should be configured?
> Or is there no way to get away from Kryo overhead?
>
> Best regards,
> Dmitry
>

Mime
View raw message