flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From StefanRRichter <...@git.apache.org>
Subject [GitHub] flink issue #5239: [FLINK-8360] Implement task-local state recovery
Date Tue, 09 Jan 2018 19:56:08 GMT
Github user StefanRRichter commented on the issue:

    Thanks for going through the general design @StephanEwen ! As we discussed, I agree with
your first point. For the second point about RocksDB, this PR already contains an optimized
way to deal with incremental local checkpoints that we did not discuss in our review, because
I thought it is too much of a low level detail.
    It does not work with duplicating streams. Instead, I introduced a state handle type for
a local directory. In fact, I mapped the previous incremental recovery from DFS state also
to this new handle type: dfs state is first downloaded and then it also simply becomes a local
directory state handle. From there, both incremental recovery paths are identical.


View raw message