flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-9169) NPE when restoring from old savepoint and state serializer could not be deserialized
Date Thu, 03 May 2018 11:51:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-9169?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16462321#comment-16462321
] 

ASF GitHub Bot commented on FLINK-9169:
---------------------------------------

Github user StefanRRichter commented on a diff in the pull request:

    https://github.com/apache/flink/pull/5950#discussion_r185771830
  
    --- Diff: flink-core/src/main/java/org/apache/flink/api/common/typeutils/TypeSerializerSerializationUtil.java
---
    @@ -69,7 +69,7 @@
     	 * written using {@link #writeSerializer(DataOutputView, TypeSerializer)}.
     	 *
     	 * <p>If deserialization fails for any reason (corrupted serializer bytes, serializer
class
    -	 * no longer in classpath, serializer class no longer valid, etc.), {@code null} will
    +	 * no longer in classpath, serializer class no longer valid, etc.), an {@link IOException}
is thrown.
    --- End diff --
    
    Comment in the next line should be deleted, looks like leftover from the copy-paste.


> NPE when restoring from old savepoint and state serializer could not be deserialized
> ------------------------------------------------------------------------------------
>
>                 Key: FLINK-9169
>                 URL: https://issues.apache.org/jira/browse/FLINK-9169
>             Project: Flink
>          Issue Type: Bug
>          Components: State Backends, Checkpointing
>    Affects Versions: 1.5.0, 1.4.2
>            Reporter: Till Rohrmann
>            Assignee: Tzu-Li (Gordon) Tai
>            Priority: Major
>             Fix For: 1.5.0
>
>
> A user reported to have observed the following exception when restoring a Flink job from
a 1.3 savepoint with Flink 1.4.
> {code}
> 2018-04-02 21:44:18,146 INFO  org.apache.flink.runtime.taskmanager.Task             
       - ApplyAMUpdate (13/160) (7248adb0b85f4458ae4144963d65
> 6fa6) switched from RUNNING to FAILED.
> java.lang.IllegalStateException: Could not initialize keyed state backend.
>         at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initKeyedState(AbstractStreamOperator.java:293)
>         at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initializeState(AbstractStreamOperator.java:225)
>         at org.apache.flink.streaming.runtime.tasks.StreamTask.initializeOperators(StreamTask.java:692)
>         at org.apache.flink.streaming.runtime.tasks.StreamTask.initializeState(StreamTask.java:679)
>         at org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(StreamTask.java:253)
>         at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
>         at java.lang.Thread.run(Thread.java:748)
> Caused by: java.lang.NullPointerException
>         at org.apache.flink.util.Preconditions.checkNotNull(Preconditions.java:58)
>         at org.apache.flink.runtime.state.RegisteredKeyedBackendStateMetaInfo.<init>(RegisteredKeyedBackendStateMetaInfo.java:53)
>         at org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend$RocksDBFullRestoreOperation.restoreKVStateMetaData(RocksDBKeyedStateB
> ackend.java:1216)
>         at org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend$RocksDBFullRestoreOperation.restoreKeyGroupsInStateHandle(RocksDBKeye
> dStateBackend.java:1153)
>         at org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend$RocksDBFullRestoreOperation.doRestore(RocksDBKeyedStateBackend.java:1
> 139)
>         at org.apache.flink.contrib.streaming.state.RocksDBKeyedStateBackend.restore(RocksDBKeyedStateBackend.java:1034)
>         at org.apache.flink.streaming.runtime.tasks.StreamTask.createKeyedStateBackend(StreamTask.java:773)
>         at org.apache.flink.streaming.api.operators.AbstractStreamOperator.initKeyedState(AbstractStreamOperator.java:283)
>         ... 6 more
> {code}
> Looking at the {{KeyedBackendStateMetaInfoSnapshotReaderWriters}}, we create {{RegisteredKeyedBackendStateMetaInfo.Snapshot}}
where the {{stateSerializer}} can be {{null}}. This is not the problem, however, in {{RocksDBKeyedStateBackend#restoreKVStateMetaData}}
we create a {{RegisteredKeyedBackendStateMetaInfo}} from the deserialized {{Snapshot}} where
we null check the state serializer. This will then fail with an indescriptive NPE.
> I think the same should happen when resuming with Flink 1.5 from a 1.4 savepoint.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message