flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Hegde, Mahendra" <Mahendra.He...@arity.com>
Subject How Flink Kafka Consumer works when it restarts
Date Wed, 12 Feb 2020 16:50:29 GMT
Hi All,

I am bit confused on Flink kafka consumer working.
I read that Flink stores the kafka message offset in checkpoint and uses it in case if it
restarts.

Question is when exactly Flink is committing about successful consumption confirmation to
kafka broker?
And when Flink job restarts will it send last offset which is available in checkpoint to kafka
broker to start consuming from that point ?
Or Kafka broker will resume based on last committed offset information available?
(I mean who manages the actual offset here, Kafka broker or the Flink client)

Thanks
Mahendra
Mime
View raw message