flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From natasky <nata...@gmail.com>
Subject Window-join DataStream (or KeyedStream) with a broadcast stream
Date Sun, 24 Nov 2019 13:20:23 GMT
Hi all,

I use window aggregation to create a stream of aggregated data per user, per
some interval.

Additionally, I use same windows to aggregate system-wide data per the same
interval.

I.e.:

Per user stream: events keyed by user ID -> tumbling window -> aggregation

System wide stream: events -> tumbling window (windowAll) -> aggregation

I need to produce a value per user, per interval, that depends on the
aggregated
data from that user and the system wide data aggregated for the
corresponding
interval.

I couldn't find a way to acheive this with Flink's windows. I think can I
get
it to work with broadcast, connect and CoProcessFunction - is that the way
to
go? How would I handle late events that way?

Thanks!
- Nathan



--
Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

Mime
View raw message