flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: Checkpoint for exact-once proccessing
Date Wed, 13 Jan 2016 10:17:22 GMT

I think there is a misunderstanding. There are no identifiers maintained
and no individual records deleted.

On recovery, all operators reset their state to a consistent snapshot:


On Wed, Jan 13, 2016 at 11:08 AM, Don Frascuchon <frascuchon@gmail.com>

> Hello,
> I'm trying to understand the process of checkpoint processing for
> exact-once in Flink, and I have some doubts.
> The documentation says that when there is a failure and the state of an
> operator is restored, the already processed records are deleted based on
> their identifiers.
> My doubts is, how these identifiers between two checkpoints are
> maintained? Every time a new input record comes to the stateful operator,
> Flink persists it before making the checkpoint? Otherwise, there may be
> messages to reprocess after a failure.
> Thanks !!!

View raw message