flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From StephanEwen <...@git.apache.org>
Subject [GitHub] flink pull request #2920: [FLINK-5218] [state backends] Eagerly close checkp...
Date Thu, 01 Dec 2016 18:41:35 GMT
GitHub user StephanEwen opened a pull request:


    [FLINK-5218] [state backends] Eagerly close checkpoint streams on cancellation

    When a task is canceled during a checkpoint operation, the operation needs to cancel fast.
    This is a forward fis from version 1.1, where checkpoints could get stuck when the state
output streams did not handle interruptions correctly (HDFS has that problem).
    Most of this is already handled in version 1.2 via the *CloseableRegistry*.
    This adds a test to validate this case is handled correctly and adds minor changes to
make it work reliably, like:
      - fail fast on `write()` on closed checkpoint streams
      - fail fast on `flush()` on closed checkpoint streams
      - slight optimization to save a flag in the checkpoint streams

You can merge this pull request into a Git repository by running:

    $ git pull https://github.com/StephanEwen/incubator-flink closing_validation

Alternatively you can review and apply these changes as the patch at:


To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

    This closes #2920
commit e592c098f25f97b223f07ff84cd2fd9233e36dc4
Author: Stephan Ewen <sewen@apache.org>
Date:   2016-12-01T16:12:12Z

    [FLINK-5218] [state backends] Add test that validates that Checkpoint Streams are eagerly
closed on cancellation.
    This is important for some stream implementations (such as HDFS) that do not properly
    handle thread interruption.


If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastructure@apache.org or file a JIRA ticket
with INFRA.

View raw message