flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Elias Levy <fearsome.lucid...@gmail.com>
Subject Does Flink DataStreams using combiners?
Date Fri, 12 Aug 2016 00:51:49 GMT
I am wondering if Flink makes use of combiners to pre-reduce a keyed and
windowed stream before shuffling the data among workers.

I.e. will it use a combiner in something like:

stream.flatMap {...}
      .assignTimestampsAndWatermarks(...)
      .keyBy(...)
      .timeWindow(...)
      .trigger(...)
      .sum("cnt")

or will it shuffle the keyed input before the sum reduction?

If it does make use of combiners, it would be useful to point this out in
the documentation, particularly if it only applies to certain types of
reducers, folds, etc.

Mime
View raw message