flink-issues mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (FLINK-5473) setMaxParallelism() higher than 1 is possible on non-parallel operators
Date Mon, 23 Jan 2017 12:25:27 GMT

    [ https://issues.apache.org/jira/browse/FLINK-5473?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15834376#comment-15834376
] 

ASF GitHub Bot commented on FLINK-5473:
---------------------------------------

Github user tillrohrmann commented on a diff in the pull request:

    https://github.com/apache/flink/pull/3182#discussion_r97281937
  
    --- Diff: flink-runtime/src/main/java/org/apache/flink/runtime/checkpoint/StateAssignmentOperation.java
---
    @@ -45,29 +45,37 @@
     
     	private final Logger logger;
     	private final Map<JobVertexID, ExecutionJobVertex> tasks;
    -	private final CompletedCheckpoint latest;
    +	private final Map<JobVertexID, TaskState> taskStates;
     	private final boolean allowNonRestoredState;
     
     	public StateAssignmentOperation(
     			Logger logger,
     			Map<JobVertexID, ExecutionJobVertex> tasks,
    -			CompletedCheckpoint latest,
    +			Map<JobVertexID, TaskState> taskStates,
     			boolean allowNonRestoredState) {
     
     		this.logger = logger;
     		this.tasks = tasks;
    -		this.latest = latest;
    +		this.taskStates = taskStates;
     		this.allowNonRestoredState = allowNonRestoredState;
     	}
     
     	public boolean assignStates() throws Exception {
    --- End diff --
    
    This method seems a bit lengthy. Maybe we could split it up.


> setMaxParallelism() higher than 1 is possible on non-parallel operators
> -----------------------------------------------------------------------
>
>                 Key: FLINK-5473
>                 URL: https://issues.apache.org/jira/browse/FLINK-5473
>             Project: Flink
>          Issue Type: Bug
>          Components: DataStream API
>    Affects Versions: 1.2.0
>            Reporter: Robert Metzger
>            Assignee: Stefan Richter
>
> While trying out Flink 1.2, I found out that you can set a maxParallelism higher than
1 on a non-parallel operator.
> I think we should have the same semantics as the setParallelism() method.
> Also, when setting a global maxParallelism in the execution environment, it will be set
as a default value for the non-parallel operator.
> When restoring a savepoint from 1.1, you have to set the maxParallelism to the parallelism
of the 1.1 job. Non-parallel operators will then also get the maxPar set to this value, leading
to an error on restore.
> So currently, users restoring from 1.1 to 1.2 have to manually set the maxParallelism
to 1 for all non-parallel operators.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message