flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: Checkpoint for exact-once proccessing
Date Wed, 13 Jan 2016 10:17:22 GMT
Hi!

I think there is a misunderstanding. There are no identifiers maintained
and no individual records deleted.

On recovery, all operators reset their state to a consistent snapshot:
https://ci.apache.org/projects/flink/flink-docs-release-0.10/internals/stream_checkpointing.html


Greetings,
Stephan


On Wed, Jan 13, 2016 at 11:08 AM, Don Frascuchon <frascuchon@gmail.com>
wrote:

> Hello,
>
> I'm trying to understand the process of checkpoint processing for
> exact-once in Flink, and I have some doubts.
>
> The documentation says that when there is a failure and the state of an
> operator is restored, the already processed records are deleted based on
> their identifiers.
>
> My doubts is, how these identifiers between two checkpoints are
> maintained? Every time a new input record comes to the stateful operator,
> Flink persists it before making the checkpoint? Otherwise, there may be
> messages to reprocess after a failure.
>
> Thanks !!!
>

Mime
View raw message