flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (FLINK-8522) DefaultOperatorStateBackend writes data in checkpoint that is never read.
Date Mon, 29 Jan 2018 16:29:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-8522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16343584#comment-16343584
] 

Tzu-Li (Gordon) Tai edited comment on FLINK-8522 at 1/29/18 4:28 PM:
---------------------------------------------------------------------

Don't we need to do both options?
 i.e.,
 1. We don't want to write that int anymore.
 2. Also, for compatibility reasons and for safeguarding ourselves, for previous savepoint
versions, we want to make sure to read (but ignore) that int, so that additions we add elsewhere
is not affected by it.

Also separate question:
Does only 1.4.0 savepoints contain that extra un-used int?


was (Author: tzulitai):
Don't we need to do both options?
 i.e.,
 1. We don't want to write that int anymore.
 2. Also, for compatibility reasons and for safeguarding ourselves, for previous savepoint
versions, we want to make sure to read (but ignore) that int, so that additions we add elsewhere
is not affected by it.

> DefaultOperatorStateBackend writes data in checkpoint that is never read.
> -------------------------------------------------------------------------
>
>                 Key: FLINK-8522
>                 URL: https://issues.apache.org/jira/browse/FLINK-8522
>             Project: Flink
>          Issue Type: Bug
>          Components: State Backends, Checkpointing
>    Affects Versions: 1.4.0
>            Reporter: Kostas Kloudas
>            Assignee: Kostas Kloudas
>            Priority: Major
>             Fix For: 1.4.1
>
>
> In the DefaultOperatorStateBackend at line 283 we write in the checkpoint an int declaring
the number of the operator states that we include in the checkpoint. 
> This number is never read when restoring and this can lead to confusion and problems
with backwards compatibility and/or extension of the types of state we support (e.g. broadcast
state).
> There are two easy solutions, either remove the line and do not write the size, or make
sure that we also read this number when restoring and simply ignore it.
> I would go for the first one, i.e. remove the line. What do you think [~richtesn] and
[~tzulitai] ?



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message