kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Development <...@yeralin.net>
Subject [DISCUSS] KIP-466: Add support for List<T> serialization and deserialization
Date Mon, 06 May 2019 16:11:02 GMT
Hello,

Starting a discussion for KIP-466 adding support for List Serde. PR is created under https://github.com/apache/kafka/pull/6592
<https://github.com/apache/kafka/pull/6592>

There are two topics I would like to discuss:
1. Since type for List serve needs to be declared before hand, I could not create a static
method for List Serde under org.apache.kafka.common.serialization.Serdes. I addressed it in
the KIP: 
P.S. Static method corresponding to ListSerde under org.apache.kafka.common.serialization.Serdes
(something like static public Serde<List<T>> List() {...} inorg.apache.kafka.common.serialization.Serdes)
class cannot be added because type needs to be defined beforehand. That's why one needs to
create List Serde in the following fashion:
new Serdes.ListSerde<String>(Serdes.String(), Comparator.comparing(String::length));
(can possibly be simplified by declaring import static org.apache.kafka.common.serialization.Serdes.ListSerde)

2. @miguno Michael G. Noll <https://github.com/miguno> is questioning whether I need
to pass a comparator to ListDeserializer. This certainly is not required. Feel free to add
your input:
https://github.com/apache/kafka/pull/6592#discussion_r281152067

Thank you!

Best,
Daniyar Yeralin

> On May 6, 2019, at 11:59 AM, Daniyar Yeralin (JIRA) <jira@apache.org> wrote:
> 
> Daniyar Yeralin created KAFKA-8326:
> --------------------------------------
> 
>             Summary: Add List<T> Serde
>                 Key: KAFKA-8326
>                 URL: https://issues.apache.org/jira/browse/KAFKA-8326
>             Project: Kafka
>          Issue Type: Improvement
>          Components: clients, streams
>            Reporter: Daniyar Yeralin
> 
> 
> I propose adding serializers and deserializers for the java.util.List class.
> 
> I have many use cases where I want to set the key of a Kafka message to be a UUID. Currently,
I need to turn UUIDs into strings or byte arrays and use their associated Serdes, but it would
be more convenient to serialize and deserialize UUIDs directly.
> 
> I believe there are many use cases where one would want to have a List serde. Ex. [https://stackoverflow.com/questions/41427174/aggregate-java-objects-in-a-list-with-kafka-streams-dsl-windows],
[https://stackoverflow.com/questions/46365884/issue-with-arraylist-serde-in-kafka-streams-api]
> 
>  
> 
> KIP Link: [https://cwiki.apache.org/confluence/display/KAFKA/KIP-466%3A+Add+support+for+List%3CT%3E+serialization+and+deserialization]
> 
> 
> 
> --
> This message was sent by Atlassian JIRA
> (v7.6.3#76005)


Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message