flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Piotr Nowojski <pi...@data-artisans.com>
Subject Re: Sink buffering
Date Wed, 04 Oct 2017 10:29:38 GMT
What do you mean by "This always depends on checkpointing interval right?”?

In TwoPhaseCommitSinkFunction, transactions are being committed on each Flink checkpoint.
I guess same applies to GenericWriteAheadSink. The first one just commits/pre-commits the
data on checkpoint, second rewrites them from the state.

If your sink supports transactions (and especially it can handle committing same transaction
twice), then both should be able to provide exactly-once.

Piotrek

> On Oct 4, 2017, at 10:30 AM, nragon <nuno.goncalves@wedotechnologies.com> wrote:
> 
> Thanks for you opinion on this.
> TwoPhaseCommitSinkFunction would probably be the best solution overall.
> Using this with something like Phoenix or Tephra would probably work. 
> This always depends on checkpointing interval right?
> 
> 
> 
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/


Mime
View raw message