flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2713) Custom StateCheckpointers should be included in the snapshots
Date Mon, 21 Sep 2015 09:37:04 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2713?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14900439#comment-14900439

ASF GitHub Bot commented on FLINK-2713:

Github user gyfora commented on the pull request:

    Another possibility which I am going to try, is to make the state restore lazy, so it
is only restored when the user calls state.value() at which point the StateCheckpointer should
be set by the user.

> Custom StateCheckpointers should be included in the snapshots
> -------------------------------------------------------------
>                 Key: FLINK-2713
>                 URL: https://issues.apache.org/jira/browse/FLINK-2713
>             Project: Flink
>          Issue Type: Bug
>          Components: Streaming
>            Reporter: Gyula Fora
>            Assignee: Gyula Fora
> Currently the restoreInitialState call fails when the user uses a custom StateCheckpointer
to create the snapshot, because the state is restored before the StateCheckpointer is set
for the StreamOperatorState. (because the restoreInitialState() call precedes the open() call)
> To avoid this issue, the custom StateCheckpointer instance should be stored within the
snapshot and should be set in the StreamOperatorState before calling restoreState(..).
> To reduce the overhead induced by this we can do 2 optimizations:
>  - We only include custom StateCheckpointers (the default java serializer one is always
>  - We only serialize the checkpointer once and store the byte array in the snapshot

This message was sent by Atlassian JIRA

View raw message