flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-6804) Inconsistent state migration behaviour between different state backends
Date Tue, 06 Jun 2017 08:50:18 GMT

    [ https://issues.apache.org/jira/browse/FLINK-6804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16038400#comment-16038400
] 

ASF GitHub Bot commented on FLINK-6804:
---------------------------------------

Github user tzulitai commented on the issue:

    https://github.com/apache/flink/pull/4073
  
    Some CEP migration tests are failing due to FLINK-6853. Rebasing this PR to include the
fix for that.


> Inconsistent state migration behaviour between different state backends
> -----------------------------------------------------------------------
>
>                 Key: FLINK-6804
>                 URL: https://issues.apache.org/jira/browse/FLINK-6804
>             Project: Flink
>          Issue Type: Bug
>          Components: State Backends, Checkpointing, Type Serialization System
>    Affects Versions: 1.3.0, 1.4.0
>            Reporter: Till Rohrmann
>            Assignee: Tzu-Li (Gordon) Tai
>            Priority: Critical
>
> The {{MemoryStateBackend}}, {{FsStateBackend}} and {{RocksDBStateBackend}} show a different
behaviour when it comes to recovery from old state and state migration. For example, using
the {{MemoryStateBackend}} it is possible to recover pojos which now have additional fields
(at recovery time). The only caveat is that the recovered {{PojoSerializer}} will silently
drop the added fields when writing the new Pojo. In contrast, the {{RocksDBStateBackend}}
correctly recognizes that a state migration is necessary and thus fails with a {{StateMigrationException}}.
The same applies to the case where Pojo field types change. The {{MemoryStateBackend}} and
the {{FsStateBackend}} accept such a change as long as the fields still have the same length.
The {{RocksDBStateBackend}} correctly fails with a {{StateMigrationException}}.
> I think that all state backends should behave similarly and give the user the same recovery
and state migration guarantees. Otherwise, it could happen that jobs run with one state backend
but not with another (wrt semantic behaviour).



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message