flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-6804) Inconsistent state migration behaviour between different state backends
Date Mon, 12 Jun 2017 14:08:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16046595#comment-16046595

ASF GitHub Bot commented on FLINK-6804:

Github user StefanRRichter commented on the issue:

    Very good work. I think the properly addresses the mentioned issues. +1

> Inconsistent state migration behaviour between different state backends
> -----------------------------------------------------------------------
>                 Key: FLINK-6804
>                 URL: https://issues.apache.org/jira/browse/FLINK-6804
>             Project: Flink
>          Issue Type: Bug
>          Components: State Backends, Checkpointing, Type Serialization System
>    Affects Versions: 1.3.0, 1.4.0
>            Reporter: Till Rohrmann
>            Assignee: Tzu-Li (Gordon) Tai
>            Priority: Critical
>              Labels: flink-rel-1.3.1-blockers
> The {{MemoryStateBackend}}, {{FsStateBackend}} and {{RocksDBStateBackend}} show a different
behaviour when it comes to recovery from old state and state migration. For example, using
the {{MemoryStateBackend}} it is possible to recover pojos which now have additional fields
(at recovery time). The only caveat is that the recovered {{PojoSerializer}} will silently
drop the added fields when writing the new Pojo. In contrast, the {{RocksDBStateBackend}}
correctly recognizes that a state migration is necessary and thus fails with a {{StateMigrationException}}.
The same applies to the case where Pojo field types change. The {{MemoryStateBackend}} and
the {{FsStateBackend}} accept such a change as long as the fields still have the same length.
The {{RocksDBStateBackend}} correctly fails with a {{StateMigrationException}}.
> I think that all state backends should behave similarly and give the user the same recovery
and state migration guarantees. Otherwise, it could happen that jobs run with one state backend
but not with another (wrt semantic behaviour).

This message was sent by Atlassian JIRA

View raw message