flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-5265) Introduce state handle replication mode for CheckpointCoordinator
Date Fri, 13 Jan 2017 11:27:26 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5265?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15821657#comment-15821657
] 

ASF GitHub Bot commented on FLINK-5265:
---------------------------------------

Github user aljoscha commented on a diff in the pull request:

    https://github.com/apache/flink/pull/2948#discussion_r95979296
  
    --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/state/OperatorStateCheckpointOutputStream.java
---
    @@ -66,8 +66,10 @@ OperatorStateHandle closeAndGetHandle() throws IOException {
     			startNewPartition();
     		}
     
    -		Map<String, long[]> offsetsMap = new HashMap<>(1);
    -		offsetsMap.put(DefaultOperatorStateBackend.DEFAULT_OPERATOR_STATE_NAME, partitionOffsets.toArray());
    +		Map<String, OperatorStateHandle.StateMetaInfo> offsetsMap = new HashMap<>(1);
    +		OperatorStateHandle.StateMetaInfo metaInfo =
    +				new OperatorStateHandle.StateMetaInfo(partitionOffsets.toArray(), OperatorStateHandle.Mode.BROADCAST);
    --- End diff --
    
    I think this should be `SPLIT_REDISTRIBUTE` to conform to the old behaviour.


> Introduce state handle replication mode for CheckpointCoordinator
> -----------------------------------------------------------------
>
>                 Key: FLINK-5265
>                 URL: https://issues.apache.org/jira/browse/FLINK-5265
>             Project: Flink
>          Issue Type: New Feature
>          Components: State Backends, Checkpointing
>    Affects Versions: 1.2.0
>            Reporter: Stefan Richter
>            Assignee: Stefan Richter
>
> Currently, the {{CheckpointCoordinator}} only supports repartitioning of {{OperatorStateHandle}}s
based on a split-and-distribute strategy. For future state types, such as broadcast or union
state, we need a different repartitioning method that allows for replicating state handles
to all subtasks.
> This is the first step on the way to implementing broadcast and union states.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message