flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Robert Metzger (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-3427) Add watermark monitoring to JobManager web frontend
Date Sat, 14 Jan 2017 10:20:26 GMT

    [ https://issues.apache.org/jira/browse/FLINK-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15822779#comment-15822779

Robert Metzger commented on FLINK-3427:

Hi Ivan,
Sadly, the person who wanted to work on this decided not to do it. So if you are still interested,
feel free to assign it to you again.

So answer your question:
I was thinking to add a new tab for “Event time”
It will show the same plan as the “Plan” view, but with the low watermarks of each operator
(We’ll compute the low watermark for the task in the web interface based on watermarks reported
by the subtasks).
There should also be a view to see the subtasks individual watermarks and the max watermark
from the subtasks

> Add watermark monitoring to JobManager web frontend
> ---------------------------------------------------
>                 Key: FLINK-3427
>                 URL: https://issues.apache.org/jira/browse/FLINK-3427
>             Project: Flink
>          Issue Type: Improvement
>          Components: Streaming, Webfrontend
>            Reporter: Robert Metzger
> Currently, its quite hard to figure out issues with the watermarks.
> I think we can improve the situation by reporting the following information through the
metrics system:
> - Report the current low watermark for each operator (this way, you can see if one operator
is preventing the watermarks to rise)
> - Report the number of events arrived after the low watermark (users can see how accurate
the watermarks are)

This message was sent by Atlassian JIRA

View raw message