flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Congxian Qiu <qcx978132...@gmail.com>
Subject Re: Externalized checkpoints
Date Fri, 23 Aug 2019 01:54:03 GMT
Hi, Vishwas

As Zhu Zhu said, you can set "state.checkpoints.num-retained"[1] to specify
the maximum number of completed checkpoints to retain.
maybe you can also ref the external checkpoint cleanup type[2] config for
how to clean up the retained checkpoint[2]

[1]
https://ci.apache.org/projects/flink/flink-docs-master/dev/stream/state/checkpointing.html#related-config-options
[2]
https://ci.apache.org/projects/flink/flink-docs-master/ops/state/checkpoints.html#retained-checkpoints
Best,
Congxian


Zhu Zhu <reedpor@gmail.com> 于2019年8月22日周四 上午10:13写道:

> Hi Vishwas,
>
> You can configure "state.checkpoints.num-retained" to specify the max
> checkpoints to retain.
> By default it is 1.
>
> Thanks,
> Zhu Zhu
>
> Vishwas Siravara <vsiravara@gmail.com> 于2019年8月22日周四 上午6:48写道:
>
>> I am also using exactly once checkpointing mode, I have a kafka source
>> and sink so both support transactions which should allow for exactly once
>> processing. Is this the reason why there is only one checkpoint retained ?
>>
>> Thanks,
>> Vishwas
>>
>> On Wed, Aug 21, 2019 at 5:26 PM Vishwas Siravara <vsiravara@gmail.com>
>> wrote:
>>
>>> Hi peeps,
>>> I am externalizing checkpoints in S3 for my flink job and I retain them
>>> on cancellation. However when I look into my S3 bucket where the
>>> checkpoints are stored there is only 1 checkpoint at any point in time . Is
>>> this the default behavior of flink where older checkpoints are deleted when
>>> the current checkpoint completes ? Here are a few screenshots. What are
>>> your thoughts on restoring an older state which is not the previous state ?
>>>
>>> List contents of bucket at time 0
>>>
>>> Object Name: checkpoints/fb9fea316bf2d530a6fc54ea107d66d4/chk-6/6af4f345-49e0-4ae1-baae-1f7c4d71ebf4Last
modified time : Wed Aug 21 22:17:23 GMT 2019
>>> Object Name: checkpoints/fb9fea316bf2d530a6fc54ea107d66d4/chk-6/_metadataLast
modified time : Wed Aug 21 22:17:24 GMT 2019
>>>
>>> List contents of bucket at time 1
>>>
>>> Printing last modified times
>>> Object Name: checkpoints/fb9fea316bf2d530a6fc54ea107d66d4/chk-12/7cf17042-7790-4909-9252-73511d93f518Last
modified time : Wed Aug 21 22:23:24 GMT 2019
>>> Object Name: checkpoints/fb9fea316bf2d530a6fc54ea107d66d4/chk-12/_metadataLast
modified time : Wed Aug 21 22:23:24 GMT 2019
>>>
>>> Thanks,
>>>
>>> Vishwas
>>>
>>>

Mime
View raw message