flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "sunjincheng (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-4937) Add incremental group window aggregation for streaming Table API
Date Mon, 14 Nov 2016 01:22:58 GMT

    [ https://issues.apache.org/jira/browse/FLINK-4937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15662466#comment-15662466
] 

sunjincheng commented on FLINK-4937:
------------------------------------

I had thought about calling transformToAggregateFunctions explicitly before createOperatorFunctionsForAggregates
in DataSetAggregate. But in DataSetAggregate will not use transformToAggregateFunctions's
return results In addition to calling createOperatorFunctionsForAggregates. so,i hope overload
createOperatorFunctionsForAggregates. What do you think @fhueske ?

> Add incremental group window aggregation for streaming Table API
> ----------------------------------------------------------------
>
>                 Key: FLINK-4937
>                 URL: https://issues.apache.org/jira/browse/FLINK-4937
>             Project: Flink
>          Issue Type: Sub-task
>          Components: Table API & SQL
>    Affects Versions: 1.2.0
>            Reporter: Fabian Hueske
>            Assignee: sunjincheng
>
> Group-window aggregates for streaming tables are currently not done in an incremental
fashion. This means that the window collects all records and performs the aggregation when
the window is closed instead of eagerly updating a partial aggregate for every added record.
Since records are buffered, non-incremental aggregation requires more storage space than incremental
aggregation.
> The DataStream API which is used under the hood of the streaming Table API features [incremental
aggregation|https://ci.apache.org/projects/flink/flink-docs-release-1.2/dev/windows.html#windowfunction-with-incremental-aggregation]
using a {{ReduceFunction}}.
> We should add support for incremental aggregation in group-windows.
> This is a follow-up task of FLINK-4691.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message