flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fabian Hueske (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8144) Optimize the timer logic in RowTimeUnboundedOver
Date Fri, 24 Nov 2017 09:39:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16265088#comment-16265088

Fabian Hueske commented on FLINK-8144:

Did you observe this behavior in practice, i.e., while debugging, or is that an assumption
based on reading the code?

I'm asking, because a timer that is registered on {{current_watermark + 1}} fires just once
when the next watermark is received (the logical clock is only advanced by watermarks).
Also by registering multiple timers on the same timestamp, the timer gets overridden, so there
will be only one timer that fires.

I think the current implementation should behave just as expected, i.e., just go once over
the MapState when a watermark is received.

> Optimize the timer logic in RowTimeUnboundedOver
> ------------------------------------------------
>                 Key: FLINK-8144
>                 URL: https://issues.apache.org/jira/browse/FLINK-8144
>             Project: Flink
>          Issue Type: Bug
>          Components: Table API & SQL
>            Reporter: Dian Fu
>            Assignee: Dian Fu
>             Fix For: 1.5.0
> Currently the logic of {{RowTimeUnboundedOver}} is as follows:
> 1) When element comes, buffer it in MapState and and register a timer at {{current watermark
+ 1}}
> 2) When event timer triggered, scan the MapState and find the elements below the current
watermark and process it. If there are remaining elements to process, register a new timer
at {{current watermark + 1}}.
> Let's assume that watermark comes about 5 seconds later than the event on average, then
we will scan about 5000 times the MapState before actually processing the events.

This message was sent by Atlassian JIRA

View raw message