Mailing-List: contact issues-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
Date: Thu, 14 Jul 2016 13:41:20 +0000 (UTC)
From: "ASF GitHub Bot (JIRA)" <jira@apache.org>
To: issues@flink.apache.org
Message-ID: <JIRA.12978188.1465830952000.28235.1468503680455@Atlassian.JIRA>
In-Reply-To: <JIRA.12978188.1465830952000@Atlassian.JIRA>
References: <JIRA.12978188.1465830952000@Atlassian.JIRA> <JIRA.12978188.1465830952347@arcas>
Subject: [jira] [Commented] (FLINK-4067) Add version header to savepoints
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Thu, 14 Jul 2016 13:41:22 -0000


    [ https://issues.apache.org/jira/browse/FLINK-4067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15376926#comment-15376926 ] 

ASF GitHub Bot commented on FLINK-4067:
---------------------------------------

Github user uce commented on the issue:

    https://github.com/apache/flink/pull/2194
  
    After talking with @tillrohrmann and @StephanEwen, we discovered some obvious cases where restoring from 1.0 savepoints won't work in 1.1, for example if window state is involved, which is serialized in a different format since 1.0. I think the same is true for RocksDB.
    
    Should we by default throw an Exception when a user tries to restore a Flink 1.0 savepoint with 1.1? On the other hand, there  are cases where it would work as expected... but if it does not work the error message will not be very helpful.


> Add version header to savepoints
> --------------------------------
>
>                 Key: FLINK-4067
>                 URL: https://issues.apache.org/jira/browse/FLINK-4067
>             Project: Flink
>          Issue Type: Improvement
>    Affects Versions: 1.0.3
>            Reporter: Ufuk Celebi
>            Assignee: Ufuk Celebi
>             Fix For: 1.1.0
>
>
> Adding a header with version information to savepoints ensures that we can migrate savepoints between Flink versions in the future (for example when changing internal serialization formats between versions).
> After talking with Till, we propose to add the following meta data:
> - Magic number (int): identify data as savepoint
> - Version (int): savepoint version (independent of Flink version)
> - Data Offset (int): specifies at which point the actual savepoint data starts. With this, we can allow future Flink versions to add fields to the header without breaking stuff, e.g. Flink 1.1 could read savepoints of Flink 2.0.
> For Flink 1.0 savepoint support, we have to try reading the savepoints without a header before failing if we don't find the magic number.


--
This message was sent by Atlassian JIRA
(v6.3.4#6332)