flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Fabian Hueske (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-6969) Add support for deferred computation for group window aggregates
Date Wed, 21 Jun 2017 21:12:00 GMT
Fabian Hueske created FLINK-6969:

             Summary: Add support for deferred computation for group window aggregates
                 Key: FLINK-6969
                 URL: https://issues.apache.org/jira/browse/FLINK-6969
             Project: Flink
          Issue Type: New Feature
          Components: Table API & SQL
            Reporter: Fabian Hueske

Deferred computation is a strategy to deal with late arriving data and avoid updates of previous
results. Instead of computing a result as soon as it is possible (i.e., when a corresponding
watermark was received), deferred computation adds a configurable amount of slack time in
which late data is accepted before the result is compute. For example, instead of computing
a tumbling window of 1 hour at each full hour, we can add a deferred computation interval
of 15 minute to compute the result quarter past each full hour.

This approach adds latency but can reduce the number of update esp. in use cases where the
user cannot influence the generation of watermarks. It is also useful if the data is emitted
to a system that cannot update result (files or Kafka). The deferred computation interval
should be configured via the {{QueryConfig}}.

This message was sent by Atlassian JIRA

View raw message