kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paul Davidson <pdavid...@salesforce.com.INVALID>
Subject Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct
Date Thu, 02 May 2019 16:48:13 GMT
Thanks for the votes everyone! KIP-411 is now accepted with:

+3 binding votes (Randall, Jason, Gwen) , and
+3 non-binding votes (Ryanne, Arjun, Magesh)

Regards,

Paul

On Wed, May 1, 2019 at 10:07 PM Arjun Satish <arjun.satish@gmail.com> wrote:

> Good point, Gwen. We always set a non empty value for client id:
>
> https://github.com/apache/kafka/blob/2.2.0/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L668
> .
>
> But more importantly, connect client ids (for consumers, for example) were
> already of the form "consumer-[0-9]+", and from now on they will be
> "connector-consumer-[connector_name]-[0-9]+". So, at least for connect
> consumers/producers, we would have already been hitting the default quota
> limits and nothing changes for them. You can correct me if I'm missing
> something, but seems like this doesn't *break* backward compatibility?
>
> I suppose this change only gives us a better way to manage that quota
> limit.
>
> Best,
>
> On Wed, May 1, 2019 at 9:16 PM Gwen Shapira <gwen@confluent.io> wrote:
>
> > I'm confused. Surely the default quota applies on empty client IDs too?
> > otherwise it will be very difficult to enforce?
> > So setting the client name will only change something if there's already
> a
> > quota for that client?
> >
> > On the other hand, I fully support switching to "easy-to-wildcard"
> template
> > for the client id.
> >
> > On Wed, May 1, 2019 at 8:50 PM Arjun Satish <arjun.satish@gmail.com>
> > wrote:
> >
> > > I just realized that setting the client.id on the will now trigger any
> > > quota restrictions (
> > > https://kafka.apache.org/documentation/#design_quotasconfig) on the
> > > broker.
> > > It seems like this PR will enforce quota policies that will either
> > require
> > > admins to set limits for each task (since the chosen format is
> > > connector-*-id), or fallback to some default value.
> > >
> > > Maybe we should mention this in the backward compatibility section for
> > the
> > > KIP. At the same time, since there is no way atm to turn off this
> > feature,
> > > should this feature be merged and released in the upcoming v2.3? This
> is
> > > something the committers can comment better.
> > >
> > > Best,
> > >
> > >
> > > On Wed, May 1, 2019 at 5:13 PM Gwen Shapira <gwen@confluent.io> wrote:
> > >
> > > > hell yeah!
> > > > +1
> > > >
> > > >
> > > > On Fri, Apr 5, 2019 at 9:08 AM Paul Davidson
> > > > <pdavidson@salesforce.com.invalid> wrote:
> > > >
> > > > > Hi all,
> > > > >
> > > > > Since we seem to have agreement in the discussion I would like to
> > start
> > > > the
> > > > > vote on KIP-411.
> > > > >
> > > > > See:
> > > > >
> > > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-411%3A+Make+default+Kafka+Connect+worker+task+client+IDs+distinct
> > > > >
> > > > > Also see the related PR: https://github.com/apache/kafka/pull/6097
> > > > >
> > > > > Thanks to everyone who contributed!
> > > > >
> > > > > Paul
> > > > >
> > > >
> > > >
> > > > --
> > > > *Gwen Shapira*
> > > > Product Manager | Confluent
> > > > 650.450.2760 | @gwenshap
> > > > Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> > > > <http://www.confluent.io/blog>
> > > >
> > >
> >
> >
> > --
> > *Gwen Shapira*
> > Product Manager | Confluent
> > 650.450.2760 | @gwenshap
> > Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> > <http://www.confluent.io/blog>
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message