flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ufuk Celebi <...@apache.org>
Subject Re: Csv to windows?
Date Mon, 14 Nov 2016 08:41:29 GMT
I think this is independent of streaming. If you want to compute the aggregate over all keys
and data you need to do this in a single task, e.g. use a (flat)map with parallelism 1, do
the aggregation there and then broadcast to downstream operators. Does this make sense or
am I overlooking something?

On 12 November 2016 at 12:18:04, Felix Neutatz (neutatz@googlemail.com) wrote:
> > want to calculate a local aggregation for each task and then 
> combine all these local aggregates to one global aggregate and 
> push this global aggregate to all nodes and continue processing 
> the data stream. If you don't understand my description, I also 
> made some drawings of what I mean: https://docs.google.com/presentation/d/13ei6pzhwNKqNShhdNWXqJaYCG1z0Hsrxfy5sRnqun5M/edit?usp=sharing


View raw message