kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gunnar Morling <gun...@hibernate.org>
Subject Exposing additional metadata in Kafka Connect schema parameters
Date Tue, 06 Mar 2018 11:29:25 GMT
Hi,

A user of the Debezium CDC Kafka Connect connectors has asked whether we
could provide information about the original source type of captured table
columns.

Usually the type info we provide by using the Kafka Connect types and some
custom semantic types is good enough. But there are some cases where
additional type info would help: e.g. in case of MySQL, MEDIUMINT and INT
columns are transmitted as Connect Int32 (as that's the smallest type which
covers their value range). But from that, a consumer can't tell wether an
INT or MEDIUMINT column should be created in a downstream database.

Now my question is: would it be a reasonable thing for us to encode the
original column type as an additional parameter of the Kafka Connect
schemas (using a special parameter name), or would this be bending the
concept of schema parameters too much? Admittedly, this metadata would be
kind of source-specific, but I can see how it'd be beneficial in some use
cases.

Thanks for any advice,

--Gunnar

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message