flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From shravan <mysore.damoda...@microfocus.com>
Subject Re: Stop job with savepoint during graceful shutdown on a k8s cluster
Date Sat, 14 Mar 2020 11:24:48 GMT
Our understanding is to stop job with savepoint, all the task manager
will persist their state during savepoint. If a Task Manager receives a
shutdown signal while savepoint is being taken, does it complete the
savepoint before shutdown ?
[Ans ] Why task manager is shutdown suddenly? Are you saying about handling
unpredictable shutdown while taking
savepoint? In that case You can also use retained check point. In case
current checkpoint has issues because of shutdown
you will have previous checkpoint. So that you can use it. Now you will have
2 options, either savepoint/checkpoint. One of them
will always be available.
*[Followup Question]* When the processing service is shutdown say for
maintenance as it is a graceful shutdown we are looking at means to avoid
duplicates as exactly once message processing is guaranteed by our service .
We are already starting the job based on checkpoint or savepoint whichever
is the latest. When the job is started from last good checkpoint it results
in duplicates.

The job manager K8S service is configured as remote job manager address
in Task Manager. This service may not be available during savepoint,  will
this affect the communication between Task Manager and Job Manager during
savepoint ?
[Ans] you can go for HA right? Where you can run more than one jobmanager so
that one is always service is available
*[Followup Question]* As we mentioned above processing service is shut down
for maintenance.

Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

View raw message