flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Chesnay Schepler (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-5380) Number of outgoing records not reported in web interface
Date Thu, 05 Jan 2017 12:52:58 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15801275#comment-15801275

Chesnay Schepler commented on FLINK-5380:

This is a problem with detecting which operator's metrics should be used for a task.

Consider a chain A -> B. Both operators A and B measure how many records they receive and
send. Since the WebInterface however only views them as a single task we don't need all of
these; we only need the input count from A and output from B; or in other words the input
from the first operator and the output from the last.

Which metrics are reused is governed by the isChainStart/isChainEnd properties of the StreamConfig,
which are set in the in the StreamingJobGraphGenerator#createChain.

The issue is simply that I did not account for the case where a chain only consists of a single
operator; in which case we need both the in- and output metrics.

In short, adding 

if (chainableOutputs.isEmpty()) {
at the end of the 
if (currentNodeId.equals(startNodeId)) {
block should fix this issue.

> Number of outgoing records not reported in web interface
> --------------------------------------------------------
>                 Key: FLINK-5380
>                 URL: https://issues.apache.org/jira/browse/FLINK-5380
>             Project: Flink
>          Issue Type: Bug
>          Components: Webfrontend
>    Affects Versions: 1.2.0
>            Reporter: Robert Metzger
>            Assignee: Sachin Goel
>         Attachments: outRecordsNotreported.png
> The web frontend does not report any outgoing records in the web frontend.
> The amount of data in MB is reported correctly.

This message was sent by Atlassian JIRA

View raw message