kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Adam Bellemare <adam.bellem...@gmail.com>
Subject Re: [DISCUSS] KIP-437: Custom replacement for MaskField SMT
Date Sat, 16 Mar 2019 15:16:09 GMT
Hi Valeria

I am thinking that a full map function via configuration is very unlikely
to be feasible. At that point, it would be best for the user to create
their own custom transformation.

I think that since your function is indeed just an extension of masking
that it is reasonable as presented. I don't have any other concerns with
the proposal, but it would be good to hear from others.

Thanks


On Fri, Mar 15, 2019 at 11:38 AM Valeria Vasylieva <
valeria.vasylieva@gmail.com> wrote:

> Hi Adam,
>
> Thank you for your interest. Here is the list of currently supported
> transformations in Connect:
> https://kafka.apache.org/documentation/#connect_transforms.
> As I can see, there is no "map" transformation in this list and all other
> SMTs do not support functionality described in a KIP.
> I cannot find the way to achieve the same result using existing
> transformations.
> The request, described in an issue was just to add this custom masking
> functionality to the MaskField SMT, but if there is a need we can evolve
> this issue and create separate "map" transformation,
> it may be more useful but will require more effort, so it is better to do
> it as separate issue.
>
> Kind Regards,
> Valeria
>
> пт, 15 мар. 2019 г. в 17:35, Adam Bellemare <adam.bellemare@gmail.com>:
>
> > Hi Valeria
> >
> > Thanks for the KIP. I admit my knowledge on Kafka Connect transforms is a
> > bit rusty, however - Is there any other way to currently achieve this
> same
> > functionality outlined in your KIP using existing transforms?
> >
> >
> > Thanks
> >
> >
> >
> >
> >
> >
> >
> >
> > On Thu, Mar 14, 2019 at 12:05 PM Valeria Vasylieva <
> > valeria.vasylieva@gmail.com> wrote:
> >
> > > Dear community members,
> > >
> > > I would be very grateful if you leave any feedback on this KIP. It will
> > > help me to understand if change is useful or not and to decide on
> further
> > > actions.
> > >
> > > Thank you in advance,
> > > Valeria
> > >
> > > пн, 11 мар. 2019 г. в 13:20, Valeria Vasylieva <
> > > valeria.vasylieva@gmail.com
> > > >:
> > >
> > > > Hi All,
> > > >
> > > > I would like to start a discussion about adding new functionality to
> > > > MaskField SMT. The existing implementation allows to mask out any
> field
> > > > value with the null equivalent of the field type.
> > > >
> > > > I suggest to add a possibility to provide a literal replacement for
> the
> > > > field. This way you can mask out any PII info (IP, SSN etc.) with any
> > > > custom replacement.
> > > >
> > > > It is a short KIP which does not require major changes, but could
> help
> > to
> > > > make this transformation more useful for the client.
> > > >
> > > > The KIP is here:
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-437%3A+Custom+replacement+for+MaskField+SMT
> > > >
> > > > I would be glad to receive any feedback on this KIP.
> > > >
> > > > Kind Regards,
> > > > Valeria
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message