flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-6014) Allow the registration of state objects in checkpoints
Date Sat, 22 Apr 2017 13:29:04 GMT

    [ https://issues.apache.org/jira/browse/FLINK-6014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15979931#comment-15979931
] 

ASF GitHub Bot commented on FLINK-6014:
---------------------------------------

Github user StefanRRichter commented on the issue:

    https://github.com/apache/flink/pull/3524
  
    Merged. @shixiaogang can you please close this PR?


> Allow the registration of state objects in checkpoints
> ------------------------------------------------------
>
>                 Key: FLINK-6014
>                 URL: https://issues.apache.org/jira/browse/FLINK-6014
>             Project: Flink
>          Issue Type: Sub-task
>          Components: State Backends, Checkpointing
>            Reporter: Xiaogang Shi
>            Assignee: Xiaogang Shi
>
> This issue is the very first step towards incremental checkpointing. We introduce a new
state handle named {{CompositeStateHandle}} to be the base of the snapshots taken by task
components.  Known implementation may include {{KeyedStateHandle}} (for {{KeyedStateBackend}}s),
{{SubtaskState}} (for subtasks, splits of {{JobVertex}}) and {{TaskState}} (for {{JobVertex}}s).
> Each {{CompositeStateHandle}} is composed of a collection of {{StateObject}s. It should
register all its state objects in {{StateRegistry}} when its checkpoint is added into {{CompletedCheckpointStore}}
(i.e., a pending checkpoint completes or a complete checkpoint is reloaded in the recovery).

> When a completed checkpoint is moved out of the {{CompletedCheckpointStore}}, we should
not simply discard all state objects in the checkpoint. With the introduction of incremental
checkpointing, a {{StateObject}} may be referenced by different checkpoints. We should unregister
all the state objects contained in the {{StateRegistry}} first. Only those state objects that
are not referenced by any checkpoint can be deleted.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message