kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Yu <yuzhih...@gmail.com>
Subject Re: [DISCUSS] KIP-291: Have separate queues for control requests and data requests
Date Thu, 14 Jun 2018 21:33:29 GMT
bq.  that's a waste of resource if control request rate is low

I don't know if control request rate can get to 100,000, likely not. Then
using the same bound as that for data requests seems high.

On Wed, Jun 13, 2018 at 10:13 PM, Lucas Wang <lucasatucla@gmail.com> wrote:

> Hi Ted,
>
> Thanks for taking a look at this KIP.
> Let's say today the setting of "queued.max.requests" in cluster A is 1000,
> while the setting in cluster B is 100,000.
> The 100 times difference might have indicated that machines in cluster B
> have larger memory.
>
> By reusing the "queued.max.requests", the controlRequestQueue in cluster B
> automatically
> gets a 100x capacity without explicitly bothering the operators.
> I understand the counter argument can be that maybe that's a waste of
> resource if control request
> rate is low and operators may want to fine tune the capacity of the
> controlRequestQueue.
>
> I'm ok with either approach, and can change it if you or anyone else feels
> strong about adding the extra config.
>
> Thanks,
> Lucas
>
>
> On Wed, Jun 13, 2018 at 3:11 PM, Ted Yu <yuzhihong@gmail.com> wrote:
>
> > Lucas:
> > Under Rejected Alternatives, #2, can you elaborate a bit more on why the
> > separate config has bigger impact ?
> >
> > Thanks
> >
> > On Wed, Jun 13, 2018 at 2:00 PM, Dong Lin <lindong28@gmail.com> wrote:
> >
> > > Hey Luca,
> > >
> > > Thanks for the KIP. Looks good overall. Some comments below:
> > >
> > > - We usually specify the full mbean for the new metrics in the KIP. Can
> > you
> > > specify it in the Public Interface section similar to KIP-237
> > > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 237%3A+More+Controller+Health+Metrics>
> > > ?
> > >
> > > - Maybe we could follow the same pattern as KIP-153
> > > <https://cwiki.apache.org/confluence/display/KAFKA/KIP-
> > > 153%3A+Include+only+client+traffic+in+BytesOutPerSec+metric>,
> > > where we keep the existing sensor name "BytesInPerSec" and add a new
> > sensor
> > > "ReplicationBytesInPerSec", rather than replacing the sensor name "
> > > BytesInPerSec" with e.g. "ClientBytesInPerSec".
> > >
> > > - It seems that the KIP changes the semantics of the broker config
> > > "queued.max.requests" because the number of total requests queued in
> the
> > > broker will be no longer bounded by "queued.max.requests". This
> probably
> > > needs to be specified in the Public Interfaces section for discussion.
> > >
> > >
> > > Thanks,
> > > Dong
> > >
> > >
> > > On Wed, Jun 13, 2018 at 12:45 PM, Lucas Wang <lucasatucla@gmail.com>
> > > wrote:
> > >
> > > > Hi Kafka experts,
> > > >
> > > > I created KIP-291 to add a separate queue for controller requests:
> > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-291%
> > > > 3A+Have+separate+queues+for+control+requests+and+data+requests
> > > >
> > > > Can you please take a look and let me know your feedback?
> > > >
> > > > Thanks a lot for your time!
> > > > Regards,
> > > > Lucas
> > > >
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message