apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bright Chen <bri...@datatorrent.com>
Subject Suggestion on optimise kryo Output
Date Tue, 10 Jan 2017 00:10:51 GMT

The kryo Output has some limitation

   - The size of the data is limited. kryo write data to a fixed buffer, it
   will throw the overflow exception if the data exceed the size
   - The Output.toBytes() will copy the data to temporary buffer and
   output, it will decrease the performance and introduce garbage collection.

When I was tuning Spillable Data structure and Manage State. I create a
mechanism to share and reuse the memory to avoid above problem. And it can
be reused in core serialization with small change. Please see jira:

Any suggestion or comments, please put in jira:



  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message