samza-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Riccomini <criccom...@apache.org>
Subject Re: Measuring Samza Job Throughput
Date Wed, 17 Jun 2015 06:26:24 GMT
Hmm, correction. I think this has to be done at tbhe KafkaSystem level. We
allow consumers and producers to return non-byte messages, which means
nothing in container can safely assume that a message is a byte array
except the serde manager. I took a look there but didn't see any byte
throughout metrics after all.

On Tuesday, June 16, 2015, Chris Riccomini <criccomini@apache.org> wrote:

> Hey Milinda,
>
> Specifically, for bytes/sec, you might want to look at serde metrics. I
> believe the serde manager tracks bytes serialized and deserialized per
> second. The consumers and producers also do this for Kafka, but on a more
> granular basis. If you want container-level throughput, serde manager is
> worth looking at.
>
> Cheers,
> Chris
>
> On Tuesday, June 16, 2015, Milinda Pathirage <mpathira@umail.iu.edu
> <javascript:_e(%7B%7D,'cvml','mpathira@umail.iu.edu');>> wrote:
>
>> Hi Devs,
>>
>> I was looking for a way to measure Samza job throughput and found that its
>> possible to do it via Samza's metrics reporter. But there several types of
>> metrics reported via this method. For example, TaskInstanceMetrics reports
>> number of messages sent. But if I wanted to get a measurement like bytes
>> per second produced, is there a way to do that. It looks
>> like KafkaSystemProducerMetrics and TaskInstanceMetrics only provide
>> number
>> of messages sent.
>>
>> If any of you have any experience in measuring Samza job throughput, can
>> you please share. Really appreciate any ideas on measuring job throughput.
>>
>> Thanks
>> Milinda
>> --
>> Milinda Pathirage
>>
>> PhD Student | Research Assistant
>> School of Informatics and Computing | Data to Insight Center
>> Indiana University
>>
>> twitter: milindalakmal
>> skype: milinda.pathirage
>> blog: http://milinda.pathirage.org
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message