flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-7213) Introduce state management by OperatorID in TaskManager
Date Mon, 14 Aug 2017 18:22:00 GMT

    [ https://issues.apache.org/jira/browse/FLINK-7213?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16126155#comment-16126155
] 

ASF GitHub Bot commented on FLINK-7213:
---------------------------------------

Github user StephanEwen commented on a diff in the pull request:

    https://github.com/apache/flink/pull/4353#discussion_r133018189
  
    --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/StateAssignmentOperation.java
---
    @@ -185,44 +184,66 @@ private void assignAttemptState(ExecutionJobVertex executionJobVertex,
List<Oper
     					subNonPartitionableState);
     
     				// PartitionedState
    -				reAssignSubPartitionableState(newManagedOperatorStates,
    +				reAssignSubPartitionableState(
    +					newManagedOperatorStates,
     					newRawOperatorStates,
     					subTaskIndex,
     					operatorIndex,
     					subManagedOperatorState,
     					subRawOperatorState);
     
     				// KeyedState
    -				if (operatorIndex == operatorIDs.size() - 1) {
    -					subKeyedState = reAssignSubKeyedStates(operatorState,
    +				if (isHeadOperator(operatorIndex, operatorIDs)) {
    --- End diff --
    
    Do we need this check here? From the JobManager and CheckpointCoordinator side, nothing
should prevent non-head operators to have keyed state. It is just a limitation in the current
API.
    
    This check seems to "enforce" an API limitation in a more general runtime that does not
actually have a need for that restriction.


> Introduce state management by OperatorID in TaskManager
> -------------------------------------------------------
>
>                 Key: FLINK-7213
>                 URL: https://issues.apache.org/jira/browse/FLINK-7213
>             Project: Flink
>          Issue Type: Improvement
>          Components: State Backends, Checkpointing
>    Affects Versions: 1.4.0
>            Reporter: Stefan Richter
>            Assignee: Stefan Richter
>
> Flink-5892 introduced the job manager / checkpoint coordinator part of managing state
on the operator level instead of the task level by introducing explicit operator_id ->
state mappings. However, this explicit mapping was not introduced in the task manager side,
so the explicit mapping is still converted into a mapping that suits the implicit operator
chain order.
> We should also introduce explicit operator ids to state management on the task manager.



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message