kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Guozhang Wang <wangg...@gmail.com>
Subject Re: [DISCUSS] KIP-444: Augment Metrics for Kafka Streams
Date Thu, 18 Jul 2019 18:29:06 GMT
Hi Boyang,

What do you mean by `per partition latency`?

Guozhang

On Mon, Jul 1, 2019 at 9:28 AM Boyang Chen <reluctanthero104@gmail.com>
wrote:

> Hey Guozhang,
>
> do we plan to add per partition latency in this KIP?
>
> On Mon, Jul 1, 2019 at 7:08 AM Bruno Cadonna <bruno@confluent.io> wrote:
>
> > Hi Guozhang,
> >
> > Thank you for the KIP.
> >
> > 1) As far as I understand, the StreamsMetrics interface is there for
> > user-defined processors. Would it make sense to also add a method to
> > the interface to specify a sensor that records skipped records?
> >
> > 2) What are the semantics of active-task-process and standby-task-process
> >
> > 3) How do dropped-late-records and expired-window-record-drop relate
> > to each other? I guess the former is for records that fall outside the
> > grace period and the latter is for records that are processed after
> > the retention period of the window. Is this correct?
> >
> > 4) Is there an actual difference between skipped and dropped records?
> > If not, shall we unify the terminology?
> >
> > 5) What happens with removed metrics when the user sets the version of
> > "built.in.metrics.version" to 2.2-
> >
> > Best,
> > Bruno
> >
> > On Thu, Jun 27, 2019 at 6:11 PM Guozhang Wang <wangguoz@gmail.com>
> wrote:
> > >
> > > Hello folks,
> > >
> > > As 2.3 is released now, I'd like to bump up this KIP discussion again
> for
> > > your reviews.
> > >
> > >
> > > Guozhang
> > >
> > >
> > > On Thu, May 23, 2019 at 4:44 PM Guozhang Wang <wangguoz@gmail.com>
> > wrote:
> > >
> > > > Hello Patrik,
> > > >
> > > > Since we are rolling out 2.3 and everyone is busy with the release
> now
> > > > this KIP does not have much discussion involved yet and will slip
> into
> > the
> > > > next release cadence.
> > > >
> > > > This KIP itself contains several parts itself: 1. refactoring the
> > existing
> > > > metrics hierarchy to cleanup some redundancy and also get more
> > clarity; 2.
> > > > add instance-level metrics like rebalance and state metrics, as well
> as
> > > > other static metrics.
> > > >
> > > >
> > > > Guozhang
> > > >
> > > >
> > > >
> > > > On Thu, May 23, 2019 at 5:34 AM Patrik Kleindl <pkleindl@gmail.com>
> > wrote:
> > > >
> > > >> Hi Guozhang
> > > >> Thanks for the KIP, this looks very helpful.
> > > >> Could you please provide more detail on the metrics planned for the
> > state?
> > > >> We were just considering how to implement this ourselves because we
> > need
> > > >> to
> > > >> track the history of stage changes.
> > > >> The idea was to have an accumulated "seconds in state x" metric for
> > every
> > > >> state.
> > > >> The new rebalance metric might solve part of our use case, but it
is
> > > >> interesting what you have planned for the state metric.
> > > >> best regards
> > > >> Patrik
> > > >>
> > > >> On Fri, 29 Mar 2019 at 18:56, Guozhang Wang <wangguoz@gmail.com>
> > wrote:
> > > >>
> > > >> > Hello folks,
> > > >> >
> > > >> > I'd like to propose the following KIP to improve the Kafka Streams
> > > >> metrics
> > > >> > mechanism to users. This includes 1) a minor change in the public
> > > >> > StreamsMetrics API, and 2) a major cleanup on the Streams' own
> > built-in
> > > >> > metrics hierarchy.
> > > >> >
> > > >> > Details can be found here:
> > > >> >
> > > >> >
> > > >> >
> > > >>
> >
> https://cwiki.apache.org/confluence/display/KAFKA/KIP-444%3A+Augment+metrics+for+Kafka+Streams
> > > >> >
> > > >> > I'd love to hear your thoughts and feedbacks. Thanks!
> > > >> >
> > > >> > --
> > > >> > -- Guozhang
> > > >> >
> > > >>
> > > >
> > > >
> > > > --
> > > > -- Guozhang
> > > >
> > >
> > >
> > > --
> > > -- Guozhang
> >
>


-- 
-- Guozhang

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message