flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-2324) Rework partitioned state storage
Date Tue, 28 Jul 2015 09:57:06 GMT

    [ https://issues.apache.org/jira/browse/FLINK-2324?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14644159#comment-14644159
] 

ASF GitHub Bot commented on FLINK-2324:
---------------------------------------

Github user StephanEwen commented on the pull request:

    https://github.com/apache/flink/pull/937#issuecomment-125532963
  
    The `Checkpointed` variant used to be check in the settings "source", "chained" and "after
repartition". That is no longer the case, so we lost the coverage there.
    
    In genera, I think we should not adjust tests and remove anything that they currently
test. There should be a new test instead.


> Rework partitioned state storage
> --------------------------------
>
>                 Key: FLINK-2324
>                 URL: https://issues.apache.org/jira/browse/FLINK-2324
>             Project: Flink
>          Issue Type: Improvement
>            Reporter: Gyula Fora
>            Assignee: Gyula Fora
>
> Partitioned states are currently stored per-key in statehandles. This is alright for
in-memory storage but is very inefficient for HDFS. 
> The logic behind the current mechanism is that this approach provides a way to repartition
a state without fetching the data from the external storage and only manipulating handles.
> We should come up with a solution that can achieve both.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message