flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gyula Fora (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-1176) Add operator for global reduce/aggregations
Date Sun, 19 Oct 2014 18:26:33 GMT
Gyula Fora created FLINK-1176:
---------------------------------

             Summary: Add operator for global reduce/aggregations
                 Key: FLINK-1176
                 URL: https://issues.apache.org/jira/browse/FLINK-1176
             Project: Flink
          Issue Type: Improvement
          Components: Streaming
            Reporter: Gyula Fora
            Priority: Minor


Currently streaming reduce operators (like .reduce or .window(..).reduce, sum, etc.) only
allow local reduces/aggregations. While one can achieve global aggregation by setting the
parallelism of these operators to 1 it can cause a serious bottleneck in the program. 

For window/batch reduces and aggregations a global reducer can be added, with parallelism
1,  to further reduce partial results. This global reduce option could be introduced as an
optional boolean parameter for the intended operators.





--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message