flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gyula Fora (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-1176) Add operator for global reduce/aggregations
Date Sun, 19 Oct 2014 18:26:33 GMT
Gyula Fora created FLINK-1176:

             Summary: Add operator for global reduce/aggregations
                 Key: FLINK-1176
                 URL: https://issues.apache.org/jira/browse/FLINK-1176
             Project: Flink
          Issue Type: Improvement
          Components: Streaming
            Reporter: Gyula Fora
            Priority: Minor

Currently streaming reduce operators (like .reduce or .window(..).reduce, sum, etc.) only
allow local reduces/aggregations. While one can achieve global aggregation by setting the
parallelism of these operators to 1 it can cause a serious bottleneck in the program. 

For window/batch reduces and aggregations a global reducer can be added, with parallelism
1,  to further reduce partial results. This global reduce option could be introduced as an
optional boolean parameter for the intended operators.

This message was sent by Atlassian JIRA

View raw message