flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Balaji Rajagopalan <balaji.rajagopa...@olacabs.com>
Subject Re: Multiple operations on a WindowedStream
Date Fri, 01 Apr 2016 08:35:19 GMT
I had a similar use case and ended writing the aggregation logic in the
apply function, could not find any better solution.

On Fri, Apr 1, 2016 at 6:03 AM, Kanak Biscuitwala <kanak.b@hotmail.com>

> Hi,
> I would like to write something that does something like a word count, and
> then emits only the 10 highest counts for that window. Logically, I would
> want to do something like:
> stream.timeWindow(Time.of(1, TimeUnit.MINUTES), Time.of(5,
> TimeUnit.SECONDS)).sum(2).apply(getTopK(10))
> However, the window context is lost after I do the sum aggregation. Is
> there a straightforward way to express this logic in Flink 1.0? One way I
> can think of is to have a complex function in apply() that has state, but I
> would like to know if there is something a little cleaner than that.
> Thanks,
> Kanak

View raw message