flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From SHI Xiaogang <shixiaoga...@gmail.com>
Subject Re: Checkpoints very slow with high backpressure
Date Thu, 01 Jun 2017 02:19:27 GMT
Hi rhashmi

We are also experiencing slow checkpoints when there exist back pressure.
It seems there is no good method to handle back pressure now.

We work around it by setting a larger number of checkpoint timeout. The
default value is 10min. But checkpoints usually take more time to complete
when there exists back pressure.  You can set it via
`CheckpointConfig#setCheckpointTimeout()`.

Regards,
Xiaogang



2017-06-01 5:36 GMT+08:00 rhashmi <rizhashmi@hotmail.com>:

> So what is the resolution? flink consuming messages from kafka. Flink went
> down about a day ago, so now flink has to process 24 hour worth of events.
> But i hit backpressure, as of right now checkpoint are timing out. Is there
> any recommendation how to handle this situation?
>
> Seems like trigger are also not firing so no update being made to down line
> database.
>
> is there recommended approach to handle backpressure?
>
> Version Flink 1.2.
>
>
>
>
>
>
> --
> View this message in context: http://apache-flink-user-
> mailing-list-archive.2336050.n4.nabble.com/Checkpoints-
> very-slow-with-high-backpressure-tp12762p13411.html
> Sent from the Apache Flink User Mailing List archive. mailing list archive
> at Nabble.com.
>

Mime
View raw message