flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8144) Optimize the timer logic in RowTimeUnboundedOver
Date Tue, 28 Nov 2017 08:39:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16268381#comment-16268381

ASF GitHub Bot commented on FLINK-8144:

Github user fhueske commented on the issue:

    I assume you had also tried periodic watermarks in your setting. I'm curious why they
didn't work for you. 
    A periodic watermark assigner extracts the timestamp from each record and just emits a
watermark when it is asked. From a correctness point of view, this should be the same as punctuated
watermarks, just with lower overhead and higher watermark latency.

> Optimize the timer logic in RowTimeUnboundedOver
> ------------------------------------------------
>                 Key: FLINK-8144
>                 URL: https://issues.apache.org/jira/browse/FLINK-8144
>             Project: Flink
>          Issue Type: Bug
>          Components: Table API & SQL
>            Reporter: Dian Fu
>            Assignee: Dian Fu
>             Fix For: 1.5.0
> Currently the logic of {{RowTimeUnboundedOver}} is as follows:
> 1) When element comes, buffer it in MapState and and register a timer at {{current watermark
+ 1}}
> 2) When event timer triggered, scan the MapState and find the elements below the current
watermark and process it. If there are remaining elements to process, register a new timer
at {{current watermark + 1}}.
> Let's assume that watermark comes about 5 seconds later than the event on average, then
we will scan about 5000 times the MapState before actually processing the events.

This message was sent by Atlassian JIRA

View raw message