ignite-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Alexey Goncharuk (JIRA)" <j...@apache.org>
Subject [jira] [Created] (IGNITE-5821) Implement fuzzy checkpoints
Date Tue, 25 Jul 2017 07:56:00 GMT
Alexey Goncharuk created IGNITE-5821:

             Summary: Implement fuzzy checkpoints
                 Key: IGNITE-5821
                 URL: https://issues.apache.org/jira/browse/IGNITE-5821
             Project: Ignite
          Issue Type: New Feature
          Components: persistence
    Affects Versions: 2.1
            Reporter: Alexey Goncharuk
             Fix For: 2.2

Currently, we are able to run only sharp checkpoints (all committed transactions are in the
checkpoint, all non-committed are not included, all data structures are fully consistent).
This has the following disadvantages:

1) All transactions are blocked for the markCheckpointBegin call
2) We have an additional overhead for checkpoint COW buffer
3) If checkpoint buffer is exhausted, we block all transactions and synchronously wait for
the checkpoint to be finished.

There is a technique called fuzzy checkpoints:
1) We keep a WAL LSN in every dirty page
2) When a page is being flushed to disk, we sync WAL up to the LSN
3) We maintain checkpoint low watermark so that WAL does not grow indefinitely
4) WAL logging is changed in a way that does not allow data structures updates to be mixed
in WAL
5) The recovery procedure is changed to apply all physical deltas up to the end of WAL and
have consistent memory state, then logical records revert all non-committed transactions

This message was sent by Atlassian JIRA

View raw message