flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Congxian Qiu <qcx978132...@gmail.com>
Subject Re: Cron style for checkpoint
Date Thu, 21 Nov 2019 06:27:15 GMT
Hi

Currently, Flink does not support such feature, from what you describe,
does set an appropriate timeout for checkpoint can solve your problem?

Best,
Congxian


shuwen zhou <jacobc3@gmail.com> 于2019年11月21日周四 下午12:06写道:

> Hi Jiayi,
> It would be great if Flink could have a user defined interface for user to
> implement to control checkpoint behavior, at least for time related
> behavior.
> I brought up a wish on JIRA [1], perhaps it described clearly enough.
>
> [1] https://issues.apache.org/jira/browse/FLINK-14884
>
>
> On Thu, 21 Nov 2019 at 11:40, Jiayi Liao <bupt_ljy@163.com> wrote:
>
>> Hi Shuwen,
>>
>>
>> As far as I know, Flink can only support checkpoint with a fixed
>> interval.
>>
>>
>> However I think the flexible mechanism of triggering checkpoint is worth
>> working on, at least from my perspective. And it may not only be a cron
>> style. In our business scenario, the data traffic usually reaches the peek
>> of the day after 20:00, which we want to increase the interval of
>> checkpoint otherwise it’ll introduce more disk and network IO.
>>
>>
>> Just want to share something about this :)
>>
>>
>>
>> Best,
>>
>> Jiayi Liao
>>
>>
>> At 2019-11-21 10:20:47, "shuwen zhou" <jacobc3@gmail.com> wrote:
>> >Hi Community,
>> >I would like to know if there is a existing function to support cron style
>> >checkpoint?
>> >The case is, our data traffic is huge on HH:30 for each hour. We don't wont
>> >checkpoint to fall in that range of time. A cron like 15,45 * * * * to set
>> >for checkpoint would be nice. If a checkpoint is already in progress when
>> >minutes is 15 or 45, there would be a config value to trigger a new
>> >checkpoint or pass.
>> >
>> >--
>> >Best Wishes,
>> >Shuwen Zhou <http://www.linkedin.com/pub/shuwen-zhou/57/55b/599/>
>>
>>
>>
>>
>>
>
>
> --
> Best Wishes,
> Shuwen Zhou <http://www.linkedin.com/pub/shuwen-zhou/57/55b/599/>
>
>

Mime
View raw message