flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Li Wang <wangli1...@gmail.com>
Subject Re: A question regarding to the checkpoint mechanism
Date Mon, 17 Oct 2016 03:31:54 GMT
Hi Gordon,

Thanks for your prompt reply.
So do you mean when we are about to checkpoint the state of an operator, we first copy its
state and then checkpoint the copied state while the operator continues processing?


> On Oct 17, 2016, at 11:10 AM, Tzu-Li (Gordon) Tai <tzulitai@apache.org> wrote:
> Hi!
> No, the operator does not need to pause processing input records while the checkpointing
of its state is in progress.
> The checkpointing of operator state is asynchronous. The operator state does not need
to be immutable, since its a copy of the snapshot state that’s checkpointed.
> Regards,
> Gordon
> On October 17, 2016 at 10:28:34 AM, Li Wang (wangli1426@gmail.com <mailto:wangli1426@gmail.com>)
>> Hi All, 
>> Any feedback is highly appreciated. 
>> Thanks. 
>> Li 
>> > On Oct 15, 2016, at 11:17 AM, Li Wang <wangli1426@gmail.com <mailto:wangli1426@gmail.com>>
>> >  
>> > Hi all, 
>> >  
>> > As far as I know, a stateful operator will checkpoint its current state to a
persistent storage when it receives all the barrier from all of its upstream operators. My
question is that does the operator doing the checkpoint need to pause processing the input
tuples for the next batch until the checkpoint is done? If yes, will it introduce significant
processing latency when the state is large. If no, does this need the operator state to be
>> >  
>> > Thanks, 
>> > Li

View raw message