flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2802) Watermark triggered operators cannot progress with cyclic flows
Date Fri, 02 Oct 2015 16:31:26 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14941348#comment-14941348

ASF GitHub Bot commented on FLINK-2802:

GitHub user gyfora opened a pull request:


    [FLINK-2802] [streaming] Remove cyclic watermark dependencies for iterations

    This PR contains a simple change so that if timestamps are enabled, iteration sources
automatically emit a `Long.MAX_VALUE` watermark.
    While this will not help making event/processing time windows consistent across cycles
(which is inherently problematic due to the cyclic time dependency), at least these programs
will not deadlock.
    I still need to add a test to validate the behaviour.

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/gyfora/flink watermark

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #1216
commit 3671742d99fe9afdb71a00ed16f5ec47f099b0bb
Author: Gyula Fora <gyfora@apache.org>
Date:   2015-10-02T16:26:32Z

    [FLINK-2802] [streaming] Remove cyclic watermark dependencies for iterations


> Watermark triggered operators cannot progress with cyclic flows
> ---------------------------------------------------------------
>                 Key: FLINK-2802
>                 URL: https://issues.apache.org/jira/browse/FLINK-2802
>             Project: Flink
>          Issue Type: Bug
>          Components: Streaming
>            Reporter: Gyula Fora
>            Priority: Blocker
> The problem is that we can easily create a cyclic watermark (time) dependency in the
stream graph which will result in a deadlock for watermark triggered operators such as  the
> A solution to this could be to emit a Long.MAX_VALUE watermark from the iteration sources.

This message was sent by Atlassian JIRA

View raw message