flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4812) Report Watermark metrics in all operators
Date Wed, 29 Nov 2017 20:24:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16271487#comment-16271487

ASF GitHub Bot commented on FLINK-4812:

Github user zentol commented on the issue:

    yes, we _could_ setup all IO metrics in the operator chain. But that will require some
more changes there since the methods there never access 2 operators at the same time. 
    For a chain A->B, to setup the output of A you need metrics for both A and B.
    You cannot be "clever" and re-use the output metrics of A as the input metrics for B,
since this will cause odd results in forking chain scenarios where the input metrics for both
chained operators change at the same time even though they are always processed sequentially.
This would get even more funky if an operator decides that it's input/output metrics are always
supposed to be identical, at which point you have metrics for 3 operators changing at the
same time.
    But just thinking about modifying the `OperatorChain` sends shivers down my spine ❄️

> Report Watermark metrics in all operators
> -----------------------------------------
>                 Key: FLINK-4812
>                 URL: https://issues.apache.org/jira/browse/FLINK-4812
>             Project: Flink
>          Issue Type: Improvement
>          Components: Metrics
>            Reporter: Robert Metzger
>            Assignee: Chesnay Schepler
>            Priority: Critical
>             Fix For: 1.5.0
> As reported by a user, Flink does currently not export the current low watermark for
sources (http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/currentLowWatermark-metric-not-reported-for-all-tasks-td13770.html).
> This JIRA is for adding such a metric for the sources as well.

This message was sent by Atlassian JIRA

View raw message