kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jouni (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-7371) Finally deprecate org.apache.kafka.common.serialization.ExtendedSerializer
Date Sat, 01 Sep 2018 09:19:00 GMT
Jouni created KAFKA-7371:

             Summary: Finally deprecate org.apache.kafka.common.serialization.ExtendedSerializer
                 Key: KAFKA-7371
                 URL: https://issues.apache.org/jira/browse/KAFKA-7371
             Project: Kafka
          Issue Type: Wish
          Components: streams
    Affects Versions: 2.0.0
            Reporter: Jouni

As mentioned in the javadocs "Once Kafka drops support for Java 7, the serialize()} method
introduced by this interface will be added to Serializer with a default implementation so
that backwards compatibility is maintained. This interface may be deprecated once that happens.".
Support for Java 7 was already dropped in Kafka 2.0.0 but this hasn't yet happened.

The problem is that some out-of-project external serializers (for example, org.springframework.kafka.support.serializer.JsonSerializer,
quite commonly used) already do add message headers when using producer API, one of those
being __TypeId__, which contains the java class name. But when using streams DSL, there's
no way to either access or modify those headers, and according to KIP-244, they get just copied
to the sink. Also, because in RecordCollectorImpl.send there are calls

final byte[] keyBytes = keySerializer.serialize(topic, key);
final byte[] valBytes = valueSerializer.serialize(topic, value);

and not

final byte[] keyBytes = keySerializer.serialize(topic, headers, key);
final byte[] valBytes = valueSerializer.serialize(topic, headers, value);

which would be possible when the plain Serializer gets the default method added. So, currently,
there's no way for anyone to write a serializer that modifies the headers if necessary when
using streams.

In my case, the problem occurred when transforming an object from input stream to a different
type of object in output stream. Took a while to debug where did those (wrong) headers come
from, and either disable adding those headers on producer side, or as I happened to be using
Processor API, modify myself the headers in ProcessorContext.

An unfortunate side-effect of two different projects making decisions affecting each other.
Not exactly a bug in either one, but a really big nuisance to find out what's happening. I'd
prefer things working as much as possible mostly out-of-the-box. Ok, API changes must sometimes
just be made.

This message was sent by Atlassian JIRA

View raw message