flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-8144) Optimize the timer logic in RowTimeUnboundedOver
Date Fri, 24 Nov 2017 09:35:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8144?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16265073#comment-16265073

ASF GitHub Bot commented on FLINK-8144:

GitHub user dianfu opened a pull request:


    [FLINK-8144] [table] Optimize the timer logic in RowTimeUnboundedOver

    ## What is the purpose of the change
    *This pull request optimize the timer handling in RowTimeUnboundedOver. Currently the
MapState will be scanned a lot of times if the watermark arrives some seconds later than the
    ## Verifying this change
    This change is already covered by existing tests, such as *OverWindowHarnessTest.testRowTimeUnboundedRangeOver*.
    ## Does this pull request potentially affect one of the following parts:
      - Dependencies (does it add or upgrade a dependency): (no)
      - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (no)
      - The serializers: (no)
      - The runtime per-record code paths (performance sensitive): (no)
      - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing,
Yarn/Mesos, ZooKeeper: (no)
      - The S3 file system connector: (no)
    ## Documentation
      - Does this pull request introduce a new feature? (no)
      - If yes, how is the feature documented? (not applicable)

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/dianfu/flink optimize_timer_over

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #5063
commit a7d7635be1b126283573ac5b55472a79d94ac0fb
Author: Dian Fu <fudian.fd@alibaba-inc.com>
Date:   2017-11-24T09:28:02Z

    [FLINK-8144] [table] Optimize the timer logic in RowTimeUnboundedOver


> Optimize the timer logic in RowTimeUnboundedOver
> ------------------------------------------------
>                 Key: FLINK-8144
>                 URL: https://issues.apache.org/jira/browse/FLINK-8144
>             Project: Flink
>          Issue Type: Bug
>          Components: Table API & SQL
>            Reporter: Dian Fu
>            Assignee: Dian Fu
>             Fix For: 1.5.0
> Currently the logic of {{RowTimeUnboundedOver}} is as follows:
> 1) When element comes, buffer it in MapState and and register a timer at {{current watermark
+ 1}}
> 2) When event timer triggered, scan the MapState and find the elements below the current
watermark and process it. If there are remaining elements to process, register a new timer
at {{current watermark + 1}}.
> Let's assume that watermark comes about 5 seconds later than the event on average, then
we will scan about 5000 times the MapState before actually processing the events.

This message was sent by Atlassian JIRA

View raw message