flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Paolo Cristofanelli <cristofanelli.pa...@gmail.com>
Subject Latency Measurement
Date Mon, 17 Jul 2017 10:42:49 GMT

I would like to understand how to measure the latency of a record.
I have set up a simple project with a Kafka consumer that reads from a
topic and performs a simple map (with a thread sleep inside).

In order to measure the latency of this mapper I have added

After that, I was planning to access the latency through the webUI
interface but the related graph does not show any values.
I do not understand why. I was thinking that I in the graph I should
observe at least the sleep duration.

I also have another question:

I am using a count window, aggregating every 100 input records and then I
perform a map. I want to see the latency as the difference between the time
at which the output record is emitted and the arrival time of the earliest
input record.

For example, the first value arrives at x. After x +5 I all the 100 values
arrived and the system can aggregate them. Now I perform the map operation
and we emit the output record at time x+15.
I would like to obtain 15 as latency.
Do you have any suggestion on how to proceed?

Thanks for your time,
Paolo Cristofanelli

View raw message