flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From rhashmi <rizhas...@hotmail.com>
Subject Re: Use Single Sink For All windows
Date Mon, 12 Jun 2017 17:05:23 GMT
Thanks Aljoscha for your response. 

I would give a try.. 
 


1- flink call *invoke* method of SinkFunction to dispatch aggregated
information. My follow up question here is .. while snapshotState method is
in process, if sink received another update then we might have mix records,
however per document all update stop during checkpoint. i assume this works
the same way.  


https://ci.apache.org/projects/flink/flink-docs-release-1.3/internals/stream_checkpointing.html

*"As soon as the operator receives snapshot barrier n from an incoming
stream, it cannot process any further records from that stream until it has
received the barrier n from the other inputs as well. Otherwise, it would
mix records that belong to snapshot n and with records that belong to
snapshot n+1."*

*"Streams that report barrier n are temporarily set aside. Records that are
received from these streams are not processed, but put into an input
buffer".
*


2- snapshotState method call when "checkpoint is requested". is there an
interface that provide when checkpoint complete .. I meant.. I will add my
flush logic right after completion of snapshot & before flink resume the
stream. With this approach we can assure that we update state only if the
checkpoint was successful. 



--
View this message in context: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/Use-Single-Sink-For-All-windows-tp13475p13652.html
Sent from the Apache Flink User Mailing List archive. mailing list archive at Nabble.com.

Mime
View raw message