samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Milinda Pathirage <mpath...@umail.iu.edu>
Subject Re: Measuring Samza Job Throughput
Date Wed, 17 Jun 2015 18:25:57 GMT
Thank you all for the ideas. I'll have a look at KafkaSystem metrics and
SamzaContainerMetrics.

Milinda

On Wed, Jun 17, 2015 at 2:38 AM, Tao Feng <fengtao04@gmail.com> wrote:

> Hi,
>
> One metric I could think of related to Samza job throughput is the
> "process-envelop" metric listed in SamzaContainerMetrics. This counter
> get incremented whenever the container process meaningful message(
>
> https://github.com/apache/samza/blob/master/samza-core/src/main/scala/org/apache/samza/container/RunLoop.scala
> &&
>
> https://github.com/apache/samza/blob/master/samza-core/src/main/scala/org/apache/samza/container/SamzaContainerMetrics.scala
> ).
>
> But this metric is more like a "QPS" type of metric .
>
> Thanks,
> -Tao
>
> On Tue, Jun 16, 2015 at 9:11 PM, Milinda Pathirage <mpathira@umail.iu.edu>
> wrote:
>
> > Hi Devs,
> >
> > I was looking for a way to measure Samza job throughput and found that
> its
> > possible to do it via Samza's metrics reporter. But there several types
> of
> > metrics reported via this method. For example, TaskInstanceMetrics
> reports
> > number of messages sent. But if I wanted to get a measurement like bytes
> > per second produced, is there a way to do that. It looks
> > like KafkaSystemProducerMetrics and TaskInstanceMetrics only provide
> number
> > of messages sent.
> >
> > If any of you have any experience in measuring Samza job throughput, can
> > you please share. Really appreciate any ideas on measuring job
> throughput.
> >
> > Thanks
> > Milinda
> > --
> > Milinda Pathirage
> >
> > PhD Student | Research Assistant
> > School of Informatics and Computing | Data to Insight Center
> > Indiana University
> >
> > twitter: milindalakmal
> > skype: milinda.pathirage
> > blog: http://milinda.pathirage.org
> >
>



-- 
Milinda Pathirage

PhD Student | Research Assistant
School of Informatics and Computing | Data to Insight Center
Indiana University

twitter: milindalakmal
skype: milinda.pathirage
blog: http://milinda.pathirage.org

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message