kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Andrew Olson (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-4932) Add UUID Serde
Date Tue, 26 Sep 2017 18:08:00 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-4932?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Andrew Olson updated KAFKA-4932:
--------------------------------
    Description: 
I propose adding serializers and deserializers for the java.util.UUID class.

I have many use cases where I want to set the key of a Kafka message to be a UUID. Currently,
I need to turn UUIDs into strings or byte arrays and use their associated Serdes, but it would
be more convenient to serialize and deserialize UUIDs directly.

I'd propose that the serializer and deserializer use the 36-byte string representation, calling
UUID.toString and UUID.fromString, and then using the existing StringSerializer / StringDeserializer
to finish the job. We would also wrap these in a Serde and modify the streams Serdes class
to include this in the list of supported types.

Optionally, we could have the deserializer support a 16-byte representation and it would check
the size of the input byte array to determine whether it's a binary or string representation
of the UUID. It's not well defined whether the most significant bits or least significant
go first, so this deserializer would have to support only one or the other.

Similary, if the deserializer supported a 16-byte representation, there could be two variants
of the serializer, a UUIDStringSerializer and a UUIDBytesSerializer.

I would be willing to write this PR, but am looking for feedback about whether there are significant
concerns here around ambiguity of what the byte representation of a UUID should be, or if
there's desire to keep to list of built-in Serdes minimal such that a PR would be unlikely
to be accepted.

KIP Link: https://cwiki.apache.org/confluence/display/KAFKA/KIP-206%3A+Add+support+for+UUID+serialization+and+deserialization

  was:
I propose adding serializers and deserializers for the java.util.UUID class.

I have many use cases where I want to set the key of a Kafka message to be a UUID. Currently,
I need to turn UUIDs into strings or byte arrays and use their associated Serdes, but it would
be more convenient to serialize and deserialize UUIDs directly.

I'd propose that the serializer and deserializer use the 36-byte string representation, calling
UUID.toString and UUID.fromString. We would also wrap these in a Serde and modify the streams
Serdes class to include this in the list of supported types.

Optionally, we could have the deserializer support a 16-byte representation and it would check
the size of the input byte array to determine whether it's a binary or string representation
of the UUID. It's not well defined whether the most significant bits or least significant
go first, so this deserializer would have to support only one or the other.

Similary, if the deserializer supported a 16-byte representation, there could be two variants
of the serializer, a UUIDStringSerializer and a UUIDBytesSerializer.

I would be willing to write this PR, but am looking for feedback about whether there are significant
concerns here around ambiguity of what the byte representation of a UUID should be, or if
there's desire to keep to list of built-in Serdes minimal such that a PR would be unlikely
to be accepted.


> Add UUID Serde
> --------------
>
>                 Key: KAFKA-4932
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4932
>             Project: Kafka
>          Issue Type: Improvement
>          Components: clients, streams
>            Reporter: Jeff Klukas
>            Assignee: Jakub Scholz
>            Priority: Minor
>              Labels: needs-kip, newbie
>
> I propose adding serializers and deserializers for the java.util.UUID class.
> I have many use cases where I want to set the key of a Kafka message to be a UUID. Currently,
I need to turn UUIDs into strings or byte arrays and use their associated Serdes, but it would
be more convenient to serialize and deserialize UUIDs directly.
> I'd propose that the serializer and deserializer use the 36-byte string representation,
calling UUID.toString and UUID.fromString, and then using the existing StringSerializer /
StringDeserializer to finish the job. We would also wrap these in a Serde and modify the streams
Serdes class to include this in the list of supported types.
> Optionally, we could have the deserializer support a 16-byte representation and it would
check the size of the input byte array to determine whether it's a binary or string representation
of the UUID. It's not well defined whether the most significant bits or least significant
go first, so this deserializer would have to support only one or the other.
> Similary, if the deserializer supported a 16-byte representation, there could be two
variants of the serializer, a UUIDStringSerializer and a UUIDBytesSerializer.
> I would be willing to write this PR, but am looking for feedback about whether there
are significant concerns here around ambiguity of what the byte representation of a UUID should
be, or if there's desire to keep to list of built-in Serdes minimal such that a PR would be
unlikely to be accepted.
> KIP Link: https://cwiki.apache.org/confluence/display/KAFKA/KIP-206%3A+Add+support+for+UUID+serialization+and+deserialization



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message