apex-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pramod Immaneni <pra...@datatorrent.com>
Subject Re: Creating a common metrics server for internal monitoring
Date Wed, 18 Nov 2015 00:10:05 GMT
I am wondering if accounting latency at a tuple level will add too much
overhead. It might be better to do it on a conditional basis.

On Tue, Nov 17, 2015 at 3:48 PM, Ganelin, Ilya <Ilya.Ganelin@capitalone.com>
wrote:

> Specifically, I¹m interested in being able to track latencies for
> individual tuples. For example, as they move through the BufferServer or
> the CircularBuffer.
>
> On 11/17/15, 3:12 PM, "David Yan" <david@datatorrent.com> wrote:
>
> >Hi Ilya,
> >
> >The AutoMetric interface is mostly designed for application-specific
> >metrics within an operator -- things like how many frauds are detected, or
> >the dollar amount processed.
> >
> >We do have BufferServer bytes and queue tuple count (a.k.a. queue size) as
> >system metrics. If you have specific metrics that pertain to BufferServer
> >and CircularBuffer that you're interested in, please state them.  We
> >should
> >probably add them to system metrics since it's probably useful in general.
> >
> >David
> >
> >On Tue, Nov 17, 2015 at 2:58 PM, Pramod Immaneni <pramod@datatorrent.com>
> >wrote:
> >
> >> Some of them are available in system stats like port queue size and are
> >> exposed via REST and to a stats listener that you can register via the
> >>API.
> >> Some of the others you mentioned can be added.
> >>
> >> On Tue, Nov 17, 2015 at 2:52 PM, Ganelin, Ilya <
> >> Ilya.Ganelin@capitalone.com>
> >> wrote:
> >>
> >> > Hi, all ­ presently Apex supports surfacing metrics via the
> >>AutoMetrics
> >> > interface, documented here:
> >> > http://docs.datatorrent.com/autometrics/
> >> >
> >> > However, this only provides utility to monitor operators. I would
> >>like to
> >> > provide a more general latency suite - fFor example, I would like to
> >> > instrument metrics around the BufferServer or the CircularQueue. Not
> >> > everything in Apex is an operator, and there are other components of
> >>the
> >> > system around which we may want explicit measurements or metrics. Is
> >>this
> >> > something that could exist? I know other projects like Spark have
> >>similar
> >> > efforts, although they are equally underdeveloped :)
> >> > ________________________________________________________
> >> >
> >> > The information contained in this e-mail is confidential and/or
> >> > proprietary to Capital One and/or its affiliates and may only be used
> >> > solely in performance of work or services for Capital One. The
> >> information
> >> > transmitted herewith is intended only for use by the individual or
> >>entity
> >> > to which it is addressed. If the reader of this message is not the
> >> intended
> >> > recipient, you are hereby notified that any review, retransmission,
> >> > dissemination, distribution, copying or other use of, or taking of any
> >> > action in reliance upon this information is strictly prohibited. If
> >>you
> >> > have received this communication in error, please contact the sender
> >>and
> >> > delete the material from your computer.
> >> >
> >>
>
> ________________________________________________________
>
> The information contained in this e-mail is confidential and/or
> proprietary to Capital One and/or its affiliates and may only be used
> solely in performance of work or services for Capital One. The information
> transmitted herewith is intended only for use by the individual or entity
> to which it is addressed. If the reader of this message is not the intended
> recipient, you are hereby notified that any review, retransmission,
> dissemination, distribution, copying or other use of, or taking of any
> action in reliance upon this information is strictly prohibited. If you
> have received this communication in error, please contact the sender and
> delete the material from your computer.
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message