kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Randall Hauch <rha...@gmail.com>
Subject Re: [DISCUSS] KIP-475: New Metric to Measure Number of Tasks on a Connector
Date Tue, 04 Jun 2019 19:25:50 GMT
Thanks, Cyrus -- this will be quite useful. I do have a few
comments/requests.

Can you please be more specific about the public details about the metric?
What is the MBean name on which the metric will appear? For example, the AK
documentation (https://kafka.apache.org/documentation/#connect_monitoring)
defines all of the metrics an where they will appear, as does
https://cwiki.apache.org/confluence/display/KAFKA/KIP-196%3A+Add+metrics+to+Kafka+Connect+framework
.

Secondly, while a metric showing the total number of tasks is very useful,
might it be worth considering also adding metrics for the number of running
tasks, the number of paused tasks, and the number of failed tasks for a
connector. It might require using the herder's `connectorStatus(String
connectorName)` method instead, but that appears to be just as effective at
using the local snapshot of the status store cache.

Thirdly, it might be useful for the KIP to address the potential
performance impact of computing these methods. Again, IIUC, the herder
methods that the proposal mentions use the status and config stores caches
only, so the impact should be negligible.

Best regards,

Randall

On Sun, Jun 2, 2019 at 10:05 PM Ryanne Dolan <ryannedolan@gmail.com> wrote:

> Cyrus, I agree this would be useful.
>
> Ryanne
>
> On Fri, May 31, 2019, 7:10 PM Oleksandr Diachenko <odiachenko@apache.org>
> wrote:
>
> >
> >
> > On 2019/05/30 06:06:12, Cyrus Vafadari <cyrus@confluent.io> wrote:
> > > Hello Dev,
> > >
> > > I'd like to start the discussion of KIP-475: New Metric to Measure
> Number
> > > of Tasks on a Connector.
> > >
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-475%3A+New+Metric+to+Measure+Number+of+Tasks+on+a+Connector
> > >
> > > The proposal is pretty straightforward -- to add a new metric to
> Connect
> > to
> > > measure the number of tasks on a Connector. Currently, we support this
> on
> > > Worker level, so this KIP just adds another metric to support this
> > > per-connector.
> > >
> > > There is also a PR:
> > > https://github.com/apache/kafka/pull/6843
> > >
> > > Thanks,
> > >
> > > Cyrus
> > >
> >
> > Hi Cyrus,
> >
> > That sounds like a useful addition.
> >
> > Regards, Alex.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message