apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Sandesh (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (APEXCORE-478) SerDe based on the types
Date Mon, 27 Jun 2016 18:46:52 GMT

     [ https://issues.apache.org/jira/browse/APEXCORE-478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Sandesh updated APEXCORE-478:
-----------------------------
    Description: 
During deploy time, Apex already knows the type of the object to be serialized and deserialized,
so can optimize the SerDe used. I am quoting Vlad here,
"
+1 for type based serialization. Tuples in most cases are flat
records/pojo and it should be possible programmatically construct a
codec that will significantly outperform Kryo. It should also reduce
amount of data passed over the wire. I started to look in that direction
as well as Kryo serialization is one of bottlenecks that limits Apex
throughput when operators are deployed into different containers
including NODE_LOCAL case."

Link to mail thread on this discussion
http://mail-archives.apache.org/mod_mbox/apex-dev/201605.mbox/%3CCAJhKGzqxknF_7Wj2T2Vfp%3D35cziT0hTJoQTnWopET%2Bj8OgpUJw%40mail.gmail.com%3E


  was:
During deploy time, Apex already knows the type of the object to be serialized and deserialized.
I am quoting Vlad here,
"
+1 for type based serialization. Tuples in most cases are flat
records/pojo and it should be possible programmatically construct a
codec that will significantly outperform Kryo. It should also reduce
amount of data passed over the wire. I started to look in that direction
as well as Kryo serialization is one of bottlenecks that limits Apex
throughput when operators are deployed into different containers
including NODE_LOCAL case."

Link to mail thread on this discussion
http://mail-archives.apache.org/mod_mbox/apex-dev/201605.mbox/%3CCAJhKGzqxknF_7Wj2T2Vfp%3D35cziT0hTJoQTnWopET%2Bj8OgpUJw%40mail.gmail.com%3E



> SerDe based on the types
> ------------------------
>
>                 Key: APEXCORE-478
>                 URL: https://issues.apache.org/jira/browse/APEXCORE-478
>             Project: Apache Apex Core
>          Issue Type: Improvement
>            Reporter: Sandesh
>
> During deploy time, Apex already knows the type of the object to be serialized and deserialized,
so can optimize the SerDe used. I am quoting Vlad here,
> "
> +1 for type based serialization. Tuples in most cases are flat
> records/pojo and it should be possible programmatically construct a
> codec that will significantly outperform Kryo. It should also reduce
> amount of data passed over the wire. I started to look in that direction
> as well as Kryo serialization is one of bottlenecks that limits Apex
> throughput when operators are deployed into different containers
> including NODE_LOCAL case."
> Link to mail thread on this discussion
> http://mail-archives.apache.org/mod_mbox/apex-dev/201605.mbox/%3CCAJhKGzqxknF_7Wj2T2Vfp%3D35cziT0hTJoQTnWopET%2Bj8OgpUJw%40mail.gmail.com%3E



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message