kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephane Maarek <steph...@simplemachines.com.au>
Subject Re: [DISCUSS] KIP-158: Kafka Connect should allow source connectors to set topic-specific settings for new topics
Date Wed, 04 Oct 2017 05:52:29 GMT
Hi Randall,

Thanks for the KIP. I like it
What happens when the target topic is already created but the configs do not match? 
i.e. wrong RF, num partitions, or missing / additional configs? Will you attempt to apply
the necessary changes or throw an error?

Thanks!
Stephane
 

´╗┐On 24/5/17, 5:59 am, "Mathieu Fenniak" <mathieu.fenniak@replicon.com> wrote:

    Ah, yes, I see you a highlighted part that should've made this clear
    to me the first read. :-)  Much clearer now!
    
    By the way, enjoyed your Debezium talk in NYC.
    
    Looking forward to this Kafka Connect change; it will allow me to
    remove a post-deployment tool that I hacked together for the purpose
    of ensuring auto-created topics have the right config.
    
    Mathieu
    
    
    On Tue, May 23, 2017 at 11:38 AM, Randall Hauch <rhauch@gmail.com> wrote:
    > Thanks for the quick feedback, Mathieu. Yes, the first configuration rule
    > whose regex matches will be applied, and no other rules will be used. I've
    > updated the KIP to try to make this more clear, but let me know if it's
    > still not clear.
    >
    > Best regards,
    >
    > Randall
    >
    > On Tue, May 23, 2017 at 10:07 AM, Mathieu Fenniak <
    > mathieu.fenniak@replicon.com> wrote:
    >
    >> Hi Randall,
    >>
    >> Awesome, very much looking forward to this.
    >>
    >> It isn't 100% clear from the KIP how multiple config-based rules would
    >> be applied; it looks like the first configuration rule whose regex
    >> matches the topic name will be used, and no other rules will be
    >> applied.  Is that correct?  (I wasn't sure if it might cascade
    >> together multiple matching rules...)
    >>
    >> Looks great,
    >>
    >> Mathieu
    >>
    >>
    >> On Mon, May 22, 2017 at 1:43 PM, Randall Hauch <rhauch@gmail.com> wrote:
    >> > Hi, all.
    >> >
    >> > We recently added the ability for Kafka Connect to create *internal*
    >> topics
    >> > using the new AdminClient, but it still would be great if Kafka Connect
    >> > could do this for new topics that result from source connector records.
    >> > I've outlined an approach to do this in "KIP-158 Kafka Connect should
    >> allow
    >> > source connectors to set topic-specific settings for new topics".
    >> >
    >> > *https://cwiki.apache.org/confluence/display/KAFKA/KIP-
    >> 158%3A+Kafka+Connect+should+allow+source+connectors+to+
    >> set+topic-specific+settings+for+new+topics
    >> > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-
    >> 158%3A+Kafka+Connect+should+allow+source+connectors+to+
    >> set+topic-specific+settings+for+new+topics>*
    >> >
    >> > Please take a look and provide feedback. Thanks!
    >> >
    >> > Best regards,
    >> >
    >> > Randall
    >>
    



Mime
View raw message