Mailing-List: contact dev-help@samza.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@samza.apache.org
MIME-Version: 1.0
In-Reply-To: 
 <CABYbY7dPKuURuJi5=NR+Mi=90W5p5CsisSqD5jCUNdXUyJh0wA@mail.gmail.com>
References: 
 <CACwebjQ2zNycwuu4vkjzTtw1AebSFRgU4tVFZi=0SgWGzG5DXw@mail.gmail.com>
	<CABYbY7dPKuURuJi5=NR+Mi=90W5p5CsisSqD5jCUNdXUyJh0wA@mail.gmail.com>
Date: Tue, 16 Jun 2015 23:26:24 -0700
Message-ID: 
 <CABYbY7cN=8FWQgeDQ=2KTWoB4bB3TzGJFCmkNj_ioaGVJ4GOxg@mail.gmail.com>
Subject: Re: Measuring Samza Job Throughput
From: Chris Riccomini <criccomini@apache.org>
To: Chris Riccomini <criccomini@apache.org>
Cc: "dev@samza.apache.org" <dev@samza.apache.org>
Content-Type: multipart/alternative; boundary=089e0122e6aa27ffb70518b0c5e1

--089e0122e6aa27ffb70518b0c5e1
Content-Type: text/plain; charset=UTF-8

Hmm, correction. I think this has to be done at tbhe KafkaSystem level. We
allow consumers and producers to return non-byte messages, which means
nothing in container can safely assume that a message is a byte array
except the serde manager. I took a look there but didn't see any byte
throughout metrics after all.

On Tuesday, June 16, 2015, Chris Riccomini <criccomini@apache.org> wrote:

> Hey Milinda,
>
> Specifically, for bytes/sec, you might want to look at serde metrics. I
> believe the serde manager tracks bytes serialized and deserialized per
> second. The consumers and producers also do this for Kafka, but on a more
> granular basis. If you want container-level throughput, serde manager is
> worth looking at.
>
> Cheers,
> Chris
>
> On Tuesday, June 16, 2015, Milinda Pathirage <mpathira@umail.iu.edu
> <javascript:_e(%7B%7D,'cvml','mpathira@umail.iu.edu');>> wrote:
>
>> Hi Devs,
>>
>> I was looking for a way to measure Samza job throughput and found that its
>> possible to do it via Samza's metrics reporter. But there several types of
>> metrics reported via this method. For example, TaskInstanceMetrics reports
>> number of messages sent. But if I wanted to get a measurement like bytes
>> per second produced, is there a way to do that. It looks
>> like KafkaSystemProducerMetrics and TaskInstanceMetrics only provide
>> number
>> of messages sent.
>>
>> If any of you have any experience in measuring Samza job throughput, can
>> you please share. Really appreciate any ideas on measuring job throughput.
>>
>> Thanks
>> Milinda
>> --
>> Milinda Pathirage
>>
>> PhD Student | Research Assistant
>> School of Informatics and Computing | Data to Insight Center
>> Indiana University
>>
>> twitter: milindalakmal
>> skype: milinda.pathirage
>> blog: http://milinda.pathirage.org
>>
>

--089e0122e6aa27ffb70518b0c5e1--