flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fabian Hueske (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-6970) Add support for late data updates to group window aggregates
Date Wed, 21 Jun 2017 21:22:02 GMT
Fabian Hueske created FLINK-6970:
------------------------------------

             Summary: Add support for late data updates to group window aggregates
                 Key: FLINK-6970
                 URL: https://issues.apache.org/jira/browse/FLINK-6970
             Project: Flink
          Issue Type: New Feature
          Components: Table API & SQL
            Reporter: Fabian Hueske


Late arriving data is a common issue for group window aggregates. At the moment, the Table
API simply drops late arriving records. Another approach are deferred computation (FLINK-6969)
and late data updates. 

This issue proposes to add late data updates for group window aggregates. Instead of discarding
the state of a window when the result has been computed, the state is kept for a certain time
interval. If a late record for a window is received within this interval, an updated result
is emitted (and the previous result is retracted). 
This feature will require a new parameter to the {{QueryConfig}} to configure the size of
the late data interval.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message