flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Till Rohrmann <trohrm...@apache.org>
Subject Re: Externalized Checkpoints vs Periodic Checkpoints
Date Thu, 02 Feb 2017 14:31:39 GMT
Hi Yassine,

a periodic checkpoint is checkpoint which will be triggered periodically by
Flink. The checkpoint itself can have multiple properties and one of them
is whether the checkpoint is externalized or not.

An externalized checkpoint is a checkpoint for which Flink writes the meta
information into a target directory. In contrast to that, for a
non-externalized checkpoint Flink will store the checkpoint meta
information only in memory. The former has the advantage that you don't
lose the checkpoints if you shutdown your cluster. They behave similar to
savepoints and in fact savepoints are externalized checkpoints with some
more properties.

At the moment, Flink's checkpoint coordinator only retains the last
successfully completed checkpoint. This means that whenever a new
checkpoint completes then the last completed checkpoint will be discarded.
This also applies to externalized checkpoints.


On Wed, Feb 1, 2017 at 2:03 PM, Yassine MARZOUGUI <y.marzougui@mindlytix.com
> wrote:

> Hi all,
> Could someone clarify the difference between externalized checkpoints[1]
> and regular periodic checkpoints[2]?
> Moreover, I have a question regarding the retention of checkpoints: For
> regular checkpoints, does the last checkpoint discard the previous ones? If
> yes, is that the case too for the externalized checkpoints? Thank you.
> Best,
> Yassine
> [1] : https://ci.apache.org/projects/flink/flink-docs-
> release-1.2/setup/checkpoints.html
> [2] : https://ci.apache.org/projects/flink/flink-docs-
> release-1.2/dev/stream/checkpointing.html

View raw message