flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From vino yang <yanghua1...@gmail.com>
Subject Re: checkpoint always fails
Date Mon, 30 Jul 2018 07:13:21 GMT
Hi Marvin,

Since you are configuring the semantics of Exactly-Once, a task will wait
for all the barriers of multiple input channels on the input side when
performing checkpoints.
This metric reflects the inconsistent progress of all upstream execution
checkpoint tasks, and some tasks may be too slow to cause align to wait a
long time.
There are many reasons why some tasks are handled too slowly, such as keyBy
forming data skew.

Thanks, vino.

2018-07-30 12:42 GMT+08:00 Marvin777 <xymaqingxiang777@gmail.com>:

> Hi vino,
>
> I found the ' Buffered During Alignment ' term to be very large,  what
> causes this phenomenon in general?
> [image: image.png]
>
>
> Marvin777 <xymaqingxiang777@gmail.com> 于2018年7月30日周一 上午10:36写道:
>
>> Hi vino,
>>
>> the issue is FLINK-9945
>> <https://issues.apache.org/jira/browse/FLINK-9945>
>>
>> thanks.
>>
>>
>> vino yang <yanghua1127@gmail.com> 于2018年7月27日周五 下午4:22写道:
>>
>>> Hi Marvin,
>>>
>>> It seems a Checkpoint Bug which triggered your checkpoint timeout. Can
>>> you create a issue in JIRA and describe your details (such as Flink
>>> version) and attach a complete log?
>>>
>>> Thanks, vino.
>>>
>>> 2018-07-26 19:37 GMT+08:00 Marvin777 <xymaqingxiang777@gmail.com>:
>>>
>>>> Hi,vino:
>>>>
>>>> Can you give me a hint,  why the checkpoint expires.
>>>>
>>>> What causes this phenomenon in general?
>>>>
>>>> [image: image.png]
>>>>
>>>>
>>>> thanks.
>>>>
>>>> Marvin777 <xymaqingxiang777@gmail.com> 于2018年7月26日周四 下午12:22写道:
>>>>
>>>>> log
>>>>> https://issues.apache.org/jira/browse/FLINK-9945   (the exception can
>>>>> not be  repeated every time, but checkpoint failed all the time.)
>>>>>
>>>>> state Backend
>>>>> [image: image.png]
>>>>>
>>>>>
>>>>> HA configuration
>>>>> [image: image.png]
>>>>>
>>>>>
>>>>> vino yang <yanghua1127@gmail.com> 于2018年7月26日周四 上午10:22写道:
>>>>>
>>>>>> Hi Marvin,
>>>>>>
>>>>>> Thanks for reporting this issue.
>>>>>>
>>>>>> Can you share more details about the failed checkpoint, such as log,
>>>>>> exception stack trace, which statebackend used, HA configuration?
>>>>>>
>>>>>> These information can help to trace the issue.
>>>>>>
>>>>>> Thanks, vino.
>>>>>>
>>>>>> 2018-07-26 10:12 GMT+08:00 Marvin777 <xymaqingxiang777@gmail.com>:
>>>>>>
>>>>>>> Hi, all:
>>>>>>>
>>>>>>> flink job can run normally, but checkpoint always fails, like
this:
>>>>>>> [image: image.png]
>>>>>>>
>>>>>>> [image: image.png]
>>>>>>> checkpoint configuration:
>>>>>>>
>>>>>>> [image: image.png]
>>>>>>>
>>>>>>> thanks.
>>>>>>>
>>>>>>>
>>>>>>
>>>

Mime
View raw message