kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andy Coates <a...@confluent.io>
Subject Re: [DISCUSS] KIP-481: SerDe Improvements for Connect Decimal type in JSON
Date Thu, 25 Jul 2019 13:12:33 GMT
Hi Almog,

Like the KIP - I think being able to support decimals in JSON in the same
way most other systems do is a great improvement.

It's not 100% clear to me from the KIP what the current format is.  Is the
text decimal a base16 encoded number, or is it base16 encoded binary form
of the number? (I've not tried to get my head around if these two are even

One suggestion I have is to change the proposed new config to only affect
decimals stored as text, i.e. to switch between the current base16 and the
more common base10.   Then add another config to the serialzier only that
controls if decimals should be serialized as text or numeric.  The benefit
of this approach is it allows us to enhance the deserializer to
automatically handle numeric decimals even without any config having to be
set, i.e. default config in the deserializer would be able to handle
numeric decimals.  Of course, this is a two edged sword: this would make
the deserializer work out of the box with numeric decimals, (yay!), but
would be a breaking change in that things that previously failed would
suddenly start deserializing.  This is a price I'm willing to pay.

IMHO, we should then plan to switch the default of decimal serialization to
numeric, and text serialization to base 10 in the next major release.
(With upgrade notes to match). Though I know this is more contentious, I
think it moves us forward in a much more standard way that the current
encoding of decimals.

On Tue, 25 Jun 2019 at 01:03, Almog Gavra <almog@confluent.io> wrote:

> Hi Everyone!
> Kicking off discussion for a new KIP:
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-481%3A+SerDe+Improvements+for+Connect+Decimal+type+in+JSON
> For those who are interested, I have a prototype implementation that helped
> guide my design: https://github.com/agavra/kafka/pull/1
> Cheers,
> Almog

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message