flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Petr Novotnik <petr.novot...@firma.seznam.cz>
Subject union followed by timestamp assignment / watermark generation
Date Sat, 06 May 2017 13:00:46 GMT
Hello Flinkers,

Given this small example program:

> https://pastebin.com/30JbbgpH

I'd expect the output:

> one|three
> two|four

However, I consistently receive ...

> one
> two|four

... due to "three" being considered a late-comer which then gets
discarded. When I remove `assignTimestampsAndWatermarks` after the
`union` and place it separately on each of the union's inputs, i.e.
before the `union`, I get what I expect.

Now, after digging through Flink's source code, this behavior actually
seems logical to me (since the `assignTimestampsAndWatermarks` and `map`
operators form one task). Though, from a user/api perspective, it is at
least surprising.

I wanted to ask whether kind of behavior is known, indented or maybe
something to be improved to avoid the gotcha?

Many thanks in advance,

View raw message