flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephan Ewen (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-3261) Tasks should eagerly report back when they cannot start a checkpoint
Date Tue, 19 Jan 2016 14:22:39 GMT
Stephan Ewen created FLINK-3261:
-----------------------------------

             Summary: Tasks should eagerly report back when they cannot start a checkpoint
                 Key: FLINK-3261
                 URL: https://issues.apache.org/jira/browse/FLINK-3261
             Project: Flink
          Issue Type: Bug
          Components: Distributed Runtime
    Affects Versions: 0.10.1
            Reporter: Stephan Ewen
            Priority: Blocker
             Fix For: 1.0.0


With very fast checkpoint intervals (few 100 msecs), it can happen that a Task is not ready
to start a checkpoint by the time it gets the first checkpoint trigger message.

If some other tasks are ready already and commence a checkpoint, the stream alignment will
make the non-participating task wait until the checkpoint expires (default: 10 minutes).

A simple way to fix this is that tasks report back when they could not start a checkpoint.
The checkpoint coordinator can then abort that checkpoint and unblock the streams by starting
new checkpoint (where all tasks will participate).

An optimization would be to send a special "abort checkpoints barrier" that tells the barrier
buffers for stream alignment to unblock a checkpoint.




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message