flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Xiaogang Shi (JIRA)" <j...@apache.org>
Subject [jira] [Closed] (FLINK-6014) Allow the registration of state objects in checkpoints
Date Sun, 23 Apr 2017 11:42:04 GMT

     [ https://issues.apache.org/jira/browse/FLINK-6014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Xiaogang Shi closed FLINK-6014.
    Resolution: Fixed

Fixed via 218bed8b8e49b0e4c61c61f696a8f010eafea1b7 and aa21f853ab0380ec1f68ae1d0b7c8d9268da4533

> Allow the registration of state objects in checkpoints
> ------------------------------------------------------
>                 Key: FLINK-6014
>                 URL: https://issues.apache.org/jira/browse/FLINK-6014
>             Project: Flink
>          Issue Type: Sub-task
>          Components: State Backends, Checkpointing
>            Reporter: Xiaogang Shi
>            Assignee: Xiaogang Shi
> This issue is the very first step towards incremental checkpointing. We introduce a new
state handle named {{CompositeStateHandle}} to be the base of the snapshots taken by task
components.  Known implementation may include {{KeyedStateHandle}} (for {{KeyedStateBackend}}s),
{{SubtaskState}} (for subtasks, splits of {{JobVertex}}) and {{TaskState}} (for {{JobVertex}}s).
> Each {{CompositeStateHandle}} is composed of a collection of {{StateObject}s. It should
register all its state objects in {{StateRegistry}} when its checkpoint is added into {{CompletedCheckpointStore}}
(i.e., a pending checkpoint completes or a complete checkpoint is reloaded in the recovery).

> When a completed checkpoint is moved out of the {{CompletedCheckpointStore}}, we should
not simply discard all state objects in the checkpoint. With the introduction of incremental
checkpointing, a {{StateObject}} may be referenced by different checkpoints. We should unregister
all the state objects contained in the {{StateRegistry}} first. Only those state objects that
are not referenced by any checkpoint can be deleted.

This message was sent by Atlassian JIRA

View raw message