kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guozhang Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-3729) Auto-configure non-default SerDes passed alongside the topology builder
Date Fri, 15 Jul 2016 17:22:20 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-3729?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15379721#comment-15379721
] 

Guozhang Wang commented on KAFKA-3729:
--------------------------------------

Currently there are two ways to specify serde classes in Kafka Streams: 1) you can specify
a "default" serde through the StreamsConfig as "key.serde", "value.serde", etc; 2) you can
override the serde in the Streams DSL on a per-operator granularity as well, for stateful
operators like join and aggregate. If you use the overloading functions without the overridden
serde class, or specify as null, the default ones in the config are still used.

Many serializer / deserializers need to be configured before use (think: AvroSerializer),
and the Kafka Serde interface provide such API for users to specify the configuration logic.
This "configure" API will be automatically triggered for case 1) above, but not for case 2),
hence users need to do sth. like 

{code}

Serializer mySer = new MySerClass();
mySer.configure(properties);

KStream joined = stream1.join(stream2, ...., mySer);

{code}

which is not convenient; we can let the library to auto-trigger the configure function for
case 2) as well so that the above example can get ride of the user manual triggering configure
function line above.

Bharat, does this make sense to you?

>  Auto-configure non-default SerDes passed alongside the topology builder
> ------------------------------------------------------------------------
>
>                 Key: KAFKA-3729
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3729
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Fred Patton
>            Assignee: Bharat Viswanadham
>              Labels: api, newbie
>
> From Guozhang Wang:
> "Only default serdes provided through configs are auto-configured today. But we could
auto-configure other serdes passed alongside the topology builder as well."



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message