flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: a question about operatorChain.broadcastCheckpointBarrier
Date Fri, 11 Mar 2016 09:23:30 GMT
Yes, you are right. In other tasks the lock is not strictly necessary but we have one method
in the base class that covers all tasks.
> On 11 Mar 2016, at 05:04, Ma GuoWei <maguowei@outlook.com> wrote:
> 
> it is very helpful。 thanks a lot 
> 
> now i know the souce task needs the lock。
> 
> Other tasks like the StreamTask will handle broadcast and checkpoint then process element
> 
> so lock is “no use” in those tasks。
> 
> am i right?
> 
> thanks your help very much!
> 
> 
> 发自我的 iPhone
> 
>> 在 2016年3月10日,下午7:45,Aljoscha Krettek <aljoscha@apache.org>
写道:
>> 
>> Hi,
>> that call needs to be in the lock scope because we need to ensure that no element
processing/element emission happens while we forward the checkpoint barrier and perform the
checkpoint.
>> 
>> Let me illustrate with an example. Say we have a source. What can happen in the source
if the call is not in the checkpoint lock scope is the following scenario:
>> 
>> Source Forwards Checkpoint Barrier
>> Source Emits Element 13
>> Source Emits Element 14
>> Source Performs Checkpoint
>> 
>> Now, the operations downstream from the source also performs a checkpoint. In the
state of this checkpoint we don’t have the changes caused by Element 13 and Element 14 reflected.
The source checkpoint, however, says that we correctly forwarded Elements 13 and 14. Therefore
the checkpoints of the source and the downstream operations are not consistent.
>> 
>> Basically the lock ensures that nothing can happen between “Forward Checkpoint
Barrier” and “Perform Checkpoint”. So the scenarios could be this, for example:
>> 
>> Source Emits Element 13
>> Source Emits Element 14
>> Source Forwards Checkpoint Barrier
>> Source Performs Checkpoint
>> 
>> Now, both the checkpoint in the source and the checkpoints of downstream operations
correctly say that elements 13 and 14 are reflected in state changes.
>> 
>> I hope this helps but let me know if you need to know more.
>> 
>> Cheers,
>> Aljoscha
>> 
>> 
>> 
>>> On 10 Mar 2016, at 06:58, Ma GuoWei <maguowei@outlook.com> wrote:
>>> 
>>> hi,all
>>> Why does the broadcastCheckpointBarrier need in the lock scope?
>>> 
>>> thanks a lot.
>> 


Mime
View raw message