flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gyula Fóra <gyula.f...@gmail.com>
Subject Re: ContinuousEventTimeTrigger breaks coGrouped windowed streams?
Date Mon, 21 Nov 2016 08:22:16 GMT
Hi William,

I am wondering whether the ContinuousEventTimeTrigger is the best choice
here (it never cleans up the state as far as I know).

Have you tried the simple SlidingEventTimeWindows as your window function?

Cheers,
Gyula

William Saar <william@saar.se> ezt írta (időpont: 2016. nov. 19., Szo,
18:28):

> Hi!
>
> My topology below seems to work when I comment out all the lines with
> ContinuousEventTimeTrigger, but prints nothing when the line is in there.
> Can I coGroup two large time windows that use a different trigger time than
> the window size? (even if the ContinuousEventTimeTrigger doesn't work for
> coGroups, I would not expect the result to be completely silent).
>
> The streams I'm cogroupng are from 2 different Kafka sources and uses
> event time with 0 out of orderness and I'm on Flink 1.1.3, if that helps
>
> DataStream<CommonType> stream1 =
>     <stream of event type1>
>     .window(TumblingEventTimeWindows.of(Time.seconds(30)))
>     .trigger(ContinuousEventTimeTrigger.of(Time.seconds(10)))
>     .fold(...);
>
>
> DataStream<CommonType> stream2 =
>     <stream of event type2>
>     .window(TumblingEventTimeWindows.of(Time.seconds(30)))
>     .trigger(ContinuousEventTimeTrigger.of(Time.seconds(10)))
>     .fold(...);
>
>
>
> stream1.coGroup(stream2).where(...).equalTo(...)
>     .window(TumblingEventTimeWindows.of(Time.seconds(30)))
>     .trigger(ContinuousEventTimeTrigger.of(Time.seconds(10)))
>     .print()
>
> Thanks,
> William
>
>

Mime
View raw message