flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Andrey Zagrebin <azagrebin.apa...@gmail.com>
Subject Re: Restoring state from an incremental RocksDB checkpoint
Date Fri, 13 Mar 2020 21:25:48 GMT
As I understand you have already enabled retained checkpoints [1] because you can only restore
from them in case of job cancellation to restart it.
Just in case, here is also the link to docs about restoring from a retained checkpoint [2]
and how to find path to it [3].

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/state/checkpoints.html#retained-checkpoints
<https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/state/checkpoints.html#retained-checkpoints>
[2] https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/state/checkpoints.html#resuming-from-a-retained-checkpoint
<https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/state/checkpoints.html#resuming-from-a-retained-checkpoint>
[3] https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/state/checkpoints.html#directory-structure
<https://ci.apache.org/projects/flink/flink-docs-release-1.10/ops/state/checkpoints.html#directory-structure>

> On 14 Mar 2020, at 00:12, Andrey Zagrebin <azagrebin.apache@gmail.com> wrote:
> 
> Hi Yuval,
> 
> You should be able to restore from the last checkpoint by restarting the job with the
same checkpoint directory.
> An incremental part is removed only if none of retained checkpoints points to it.
> 
> Best,
> Andrey
> 
>> On 13 Mar 2020, at 16:06, Yuval Itzchakov <yuvalos@gmail.com> wrote:
>> 
>> Hi,
>> 
>> We're using RocksDB as a state backend. We've come to a situation where due to high
backpressure in one of our operators, we can't make a savepoint complete.
>> 
>> Since we have retained previous checkpoints, I was wondering if these would be eligible
to serve as a restoration point, given that we are taking advantage of RocksDBs incremental
snapshot capability, I was unsure. Would the incremental snapshot be missing data? or do they
point to the remaining parts of previous checkpoints?
> 


Mime
View raw message