flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Renjie Liu <liurenjie2...@gmail.com>
Subject Re: Question about the checkpoint mechanism in Flink.
Date Tue, 01 Nov 2016 14:21:03 GMT
Essentially you are right, but the snapshot commit process is asynchronous.
That's what you have to pay for exactly once semantics.

Li Wang <wangli1426@gmail.com>于2016年11月1日周二 下午3:05写道:

> Hi all,
>
> I have a question regarding to the state checkpoint mechanism in Flink. I
> find the statement  "Once the last stream has received barrier n, the
> operator emits all pending outgoing records, and then emits
> snapshot n barriers itself” on the document
> https://ci.apache.org/projects/flink/flink-docs-master/internals/stream_checkpointing.html#exactly-once-vs-at-least-once
> .
>
> Does this mean that to achieve exactly-once semantic, instead of sending
> tuples downstream immediately the operator buffers its outgoing tuples in a
> pending queue until the current snapshot is committed? If yes, will this
> introduce significant processing delay?
>
> Thanks,
> Li
>
> --
Liu, Renjie
Software Engineer, MVAD

Mime
View raw message