flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gyula Fora (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-2713) Custom StateCheckpointers should be included in the snapshots
Date Sat, 19 Sep 2015 21:00:05 GMT
Gyula Fora created FLINK-2713:

             Summary: Custom StateCheckpointers should be included in the snapshots
                 Key: FLINK-2713
                 URL: https://issues.apache.org/jira/browse/FLINK-2713
             Project: Flink
          Issue Type: Bug
          Components: Streaming
            Reporter: Gyula Fora

Currently the restoreInitialState call fails when the user uses a custom StateCheckpointer
to create the snapshot, because the state is restored before the StateCheckpointer is set
for the StreamOperatorState. (because the restoreInitialState() call precedes the open() call)

To avoid this issue, the custom StateCheckpointer instance should be stored within the snapshot
and should be set in the StreamOperatorState before calling restoreState(..).

To reduce the overhead induced by this we can do 2 optimizations:
 - We only include custom StateCheckpointers (the default java serializer one is always available)
 - We only serialize the checkpointer once and store the byte array in the snapshot

This message was sent by Atlassian JIRA

View raw message