flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ufuk Celebi <...@apache.org>
Subject Re: How to measure Flink performance
Date Mon, 09 May 2016 09:17:19 GMT
Hey Prateek,

On Fri, May 6, 2016 at 6:40 PM, prateekarora <prateek.arora2k6@gmail.com> wrote:
> I have below information from spark . do i can get similar information from
> Flink also ? if yes then how can i get that.

You can get GC time via the task manager overview.

The other metrics don't necessarily translate to Flink as Flink is not
executing your streaming program as mini-batches, but your program is
executed with continuous (long lived) operators.

This means for example that shuffles are continiously exchanging data
and you can't easily look at "how long the shuffle took". Also, the
scheduler delay and serialization times are not that interesting for
Flink as the cost of this is amortized over one long-running job (e.g.
because zero if your job is running long enough ;)) and you don't
schedule and serialize the tasks multiple times.

– Ufuk

View raw message