flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Seth Wiesman (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-6315) Notify on checkpoint timeout
Date Thu, 20 Apr 2017 13:36:04 GMT

    [ https://issues.apache.org/jira/browse/FLINK-6315?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15976688#comment-15976688

Seth Wiesman commented on FLINK-6315:

Thank you for talking this through with me, I'm going to close this issue and the PR. I think
I can use some of this to make the eventually consistent sink work with concurrent checkpoints;
we'll see if it works otherwise I will document the sink as only being consistent with one
concurrent checkpoint. 

> Notify on checkpoint timeout 
> -----------------------------
>                 Key: FLINK-6315
>                 URL: https://issues.apache.org/jira/browse/FLINK-6315
>             Project: Flink
>          Issue Type: New Feature
>          Components: Core
>            Reporter: Seth Wiesman
>            Assignee: Seth Wiesman
> A common use case when writing a custom operator that outputs data to some third party
location to partially output on checkpoint and then commit on notifyCheckpointComplete. If
that external system does not gracefully handle rollbacks (such as Amazon S3 not allowing
consistent delete operations) then that data needs to be handled by the next checkpoint. 
> The idea is to add a new interface similar to CheckpointListener that provides a callback
when the CheckpointCoordinator timesout a checkpoint
> {code:java}
> /**
>  * This interface must be implemented by functions/operations that want to receive
>  * a notification if a checkpoint has been {@link org.apache.flink.runtime.checkpoint.CheckpointCoordinator}
>  */
> public interface CheckpointTimeoutListener {
> 	/**
> 	 * This method is called as a notification if a distributed checkpoint has been timed
> 	 *
> 	 * @param checkpointId The ID of the checkpoint that has been timed out.
> 	 * @throws Exception
> 	 */
> 	void notifyCheckpointTimeout(long checkpointId) throws Exception;
> }
> {code}

This message was sent by Atlassian JIRA

View raw message