flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@gmail.com>
Subject Re: Introduce Barriers in stream source
Date Mon, 13 Aug 2018 15:27:17 GMT
Hi,

It is sufficient to implement the CheckpointedFunction interface.
Since SourceFunctions emit records in a separate thread, you need to ensure
that not record is emitted while the shapshotState method is called.
Flink provides a lock to synchronize data emission and state snapshotting.
See the JavaDocs of SourceFunction for details.

Best, Fabian

2018-08-13 16:42 GMT+02:00 Darshan Singh <darshan.meel@gmail.com>:

> Hi,
>
> I am implementing a source and I want to use checkpointing and would like
> to restore the job from these external checkpoints. I used Kafka for my
> tests and it worked fine.
>
> However, I would like to know if I have my own source what do I need to
> do. I am sure that I will need to implement CheckpointedFunctions
> (initializeState and snapshotState) . Based on what I read and looked at
> implemetation of Kafka source I do not need to do anything else apart from
> this.
> Maybe notifyCheckpointComplete also.
>
> But I would like to confirm if I will need to implement something to
> create Barrier in my source.
>
> Thanks
>

Mime
View raw message