flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Gyula Fora (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-2403) Rework partitioned state checkpointing logic
Date Thu, 23 Jul 2015 19:56:04 GMT
Gyula Fora created FLINK-2403:

             Summary: Rework partitioned state checkpointing logic
                 Key: FLINK-2403
                 URL: https://issues.apache.org/jira/browse/FLINK-2403
             Project: Flink
          Issue Type: Improvement
          Components: Streaming
    Affects Versions: 0.10
            Reporter: Gyula Fora

In case of partitioned states, states are stored per-key (creating a statehandle for each
key) so that we can later use this for repartitioning without fetching all the states. 

While this mechanism might be desired later it is extremely inefficient if we use a state
backend such as HDFS as we are creating many small files.

This message was sent by Atlassian JIRA

View raw message