Mailing-List: contact dev-help@samza.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@samza.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CAK+e0ouqRRgqmct-fCYdC_S1z_+-EP-SbsLpWPbtHBrDMMn3jw@mail.gmail.com>
References: 
 <CACwebjQ2zNycwuu4vkjzTtw1AebSFRgU4tVFZi=0SgWGzG5DXw@mail.gmail.com>
	<CAK+e0ouqRRgqmct-fCYdC_S1z_+-EP-SbsLpWPbtHBrDMMn3jw@mail.gmail.com>
Date: Wed, 17 Jun 2015 14:25:57 -0400
Message-ID: 
 <CACwebjRcuehaaMWHeU6Gzpznr7f8dM3eEUP3w5HegxQ1cwi1AA@mail.gmail.com>
Subject: Re: Measuring Samza Job Throughput
From: Milinda Pathirage <mpathira@umail.iu.edu>
To: dev@samza.apache.org
Content-Type: multipart/alternative; boundary=f46d0418262674424d0518bad25a

--f46d0418262674424d0518bad25a
Content-Type: text/plain; charset=UTF-8

Thank you all for the ideas. I'll have a look at KafkaSystem metrics and
SamzaContainerMetrics.

Milinda

On Wed, Jun 17, 2015 at 2:38 AM, Tao Feng <fengtao04@gmail.com> wrote:

> Hi,
>
> One metric I could think of related to Samza job throughput is the
> "process-envelop" metric listed in SamzaContainerMetrics. This counter
> get incremented whenever the container process meaningful message(
>
> https://github.com/apache/samza/blob/master/samza-core/src/main/scala/org/apache/samza/container/RunLoop.scala
> &&
>
> https://github.com/apache/samza/blob/master/samza-core/src/main/scala/org/apache/samza/container/SamzaContainerMetrics.scala
> ).
>
> But this metric is more like a "QPS" type of metric .
>
> Thanks,
> -Tao
>
> On Tue, Jun 16, 2015 at 9:11 PM, Milinda Pathirage <mpathira@umail.iu.edu>
> wrote:
>
> > Hi Devs,
> >
> > I was looking for a way to measure Samza job throughput and found that
> its
> > possible to do it via Samza's metrics reporter. But there several types
> of
> > metrics reported via this method. For example, TaskInstanceMetrics
> reports
> > number of messages sent. But if I wanted to get a measurement like bytes
> > per second produced, is there a way to do that. It looks
> > like KafkaSystemProducerMetrics and TaskInstanceMetrics only provide
> number
> > of messages sent.
> >
> > If any of you have any experience in measuring Samza job throughput, can
> > you please share. Really appreciate any ideas on measuring job
> throughput.
> >
> > Thanks
> > Milinda
> > --
> > Milinda Pathirage
> >
> > PhD Student | Research Assistant
> > School of Informatics and Computing | Data to Insight Center
> > Indiana University
> >
> > twitter: milindalakmal
> > skype: milinda.pathirage
> > blog: http://milinda.pathirage.org
> >
>


-- 
Milinda Pathirage

PhD Student | Research Assistant
School of Informatics and Computing | Data to Insight Center
Indiana University

twitter: milindalakmal
skype: milinda.pathirage
blog: http://milinda.pathirage.org

--f46d0418262674424d0518bad25a--