flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Rohrmann <trohrm...@apache.org>
Subject Re: Cannot see all events in window apply() for big input
Date Mon, 07 Nov 2016 17:17:45 GMT
And this other job also performs a window operation based on event time?

What do you mean with “I have a doubt is the necessary parallelism for
window operation if reprocessing a skew input from Kafka”?

Also be aware that the windowAll operation is executed with a dop of 1,
making it effectively a non-parallel operation.

Have you tried switching to the latest Flink version for the tests?

Cheers,
Till
​

On Mon, Nov 7, 2016 at 5:43 PM, Sendoh <unicorn.banachi@gmail.com> wrote:

> Hi Till.
>
> Thank you for suggesting. We know the timestamp is correct because another
> Flink job is running with the three topics correctly. We also know the
> operators work well before window apply() because we check the result
> before
> window apply().
>
> What currently I have a doubt is the necessary parallelisms for window
> operation if reprocessing a skew input from Kafka because it works with
> fewer events, and small topics always appear while big topics disappear.
>
> Best,
>
> Sendoh
>
>
>
> --
> View this message in context: http://apache-flink-user-
> mailing-list-archive.2336050.n4.nabble.com/Cannot-see-all-
> events-in-window-apply-for-big-input-tp9945p9950.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> at Nabble.com.
>

Mime
View raw message