kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Arjun Satish <arjun.sat...@gmail.com>
Subject Re: [VOTE] KIP-411: Make default Kafka Connect worker task client IDs distinct
Date Thu, 02 May 2019 05:07:31 GMT
Good point, Gwen. We always set a non empty value for client id:
https://github.com/apache/kafka/blob/2.2.0/clients/src/main/java/org/apache/kafka/clients/consumer/KafkaConsumer.java#L668
.

But more importantly, connect client ids (for consumers, for example) were
already of the form "consumer-[0-9]+", and from now on they will be
"connector-consumer-[connector_name]-[0-9]+". So, at least for connect
consumers/producers, we would have already been hitting the default quota
limits and nothing changes for them. You can correct me if I'm missing
something, but seems like this doesn't *break* backward compatibility?

I suppose this change only gives us a better way to manage that quota limit.

Best,

On Wed, May 1, 2019 at 9:16 PM Gwen Shapira <gwen@confluent.io> wrote:

> I'm confused. Surely the default quota applies on empty client IDs too?
> otherwise it will be very difficult to enforce?
> So setting the client name will only change something if there's already a
> quota for that client?
>
> On the other hand, I fully support switching to "easy-to-wildcard" template
> for the client id.
>
> On Wed, May 1, 2019 at 8:50 PM Arjun Satish <arjun.satish@gmail.com>
> wrote:
>
> > I just realized that setting the client.id on the will now trigger any
> > quota restrictions (
> > https://kafka.apache.org/documentation/#design_quotasconfig) on the
> > broker.
> > It seems like this PR will enforce quota policies that will either
> require
> > admins to set limits for each task (since the chosen format is
> > connector-*-id), or fallback to some default value.
> >
> > Maybe we should mention this in the backward compatibility section for
> the
> > KIP. At the same time, since there is no way atm to turn off this
> feature,
> > should this feature be merged and released in the upcoming v2.3? This is
> > something the committers can comment better.
> >
> > Best,
> >
> >
> > On Wed, May 1, 2019 at 5:13 PM Gwen Shapira <gwen@confluent.io> wrote:
> >
> > > hell yeah!
> > > +1
> > >
> > >
> > > On Fri, Apr 5, 2019 at 9:08 AM Paul Davidson
> > > <pdavidson@salesforce.com.invalid> wrote:
> > >
> > > > Hi all,
> > > >
> > > > Since we seem to have agreement in the discussion I would like to
> start
> > > the
> > > > vote on KIP-411.
> > > >
> > > > See:
> > > >
> > > >
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-411%3A+Make+default+Kafka+Connect+worker+task+client+IDs+distinct
> > > >
> > > > Also see the related PR: https://github.com/apache/kafka/pull/6097
> > > >
> > > > Thanks to everyone who contributed!
> > > >
> > > > Paul
> > > >
> > >
> > >
> > > --
> > > *Gwen Shapira*
> > > Product Manager | Confluent
> > > 650.450.2760 | @gwenshap
> > > Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> > > <http://www.confluent.io/blog>
> > >
> >
>
>
> --
> *Gwen Shapira*
> Product Manager | Confluent
> 650.450.2760 | @gwenshap
> Follow us: Twitter <https://twitter.com/ConfluentInc> | blog
> <http://www.confluent.io/blog>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message