kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Francesco Lemma (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (KAFKA-4691) ProducerInterceptor.onSend() is called after key and value are serialized
Date Wed, 25 Jan 2017 21:56:27 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15838657#comment-15838657
] 

Francesco Lemma edited comment on KAFKA-4691 at 1/25/17 9:56 PM:
-----------------------------------------------------------------

[~mjsax] This issue is not related to dynamic routing. Even though the problem I'm trying
to solve is dynamic routing, effectively the {{ProducerInterceptor}} cannot be used within
KafkaStreams to perform operations which involve the key and/or the value of the message (without
the cost at least of an additional deserialization. I say "at least" because there's potentially
also an additional serialization in case the intercepted record needs to carry a modified
version of the message).


was (Author: francescolemma):
[~mjsax] This issue is not related to dynamic routing. Even though the problem I'm trying
to solve is dynamic routing, effectively the {{ProducerInterceptor}} cannot be used within
KafkaStreams to perform operations which involve the key and/or the value of the message.

> ProducerInterceptor.onSend() is called after key and value are serialized
> -------------------------------------------------------------------------
>
>                 Key: KAFKA-4691
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4691
>             Project: Kafka
>          Issue Type: Bug
>          Components: clients, streams
>    Affects Versions: 0.10.1.1
>            Reporter: Francesco Lemma
>              Labels: easyfix
>         Attachments: 2017-01-24 00_50_55-SDG_CR33_DevStudio - Java EE - org.apache.kafka.streams.processor.internals.Reco.png
>
>
> According to the JavaDoc (https://kafka.apache.org/0101/javadoc/org/apache/kafka/clients/producer/ProducerInterceptor.html)
"	This is called from KafkaProducer.send(ProducerRecord) and KafkaProducer.send(ProducerRecord,
Callback) methods, before key and value get serialized and partition is assigned (if partition
is not specified in ProducerRecord)".
> Although when using this with Kafka Streams (StreamsConfig.producerPrefix(ProducerConfig.INTERCEPTOR_CLASSES_CONFIG))
the key and value contained in the record object are already serialized.
> As you can see from the screenshot, the serialization is performed inside RecordCollectionImpl.send(ProducerRecord<K,
V> record, Serializer<K> keySerializer, Serializer<V> valueSerializer,
>                             StreamPartitioner<K, V> partitioner), effectively before
calling the send method of the producer which will trigger the interceptor.
> This makes it unable to perform any kind of operation involving the key or value of the
message, unless at least performing an additional deserialization.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message