flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Maximilian Michels (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-3047) Local batch execution: set number of task manager slots to the maximum parallelism
Date Thu, 19 Nov 2015 11:18:11 GMT
Maximilian Michels created FLINK-3047:
-----------------------------------------

             Summary: Local batch execution: set number of task manager slots to the maximum
parallelism
                 Key: FLINK-3047
                 URL: https://issues.apache.org/jira/browse/FLINK-3047
             Project: Flink
          Issue Type: Bug
          Components: Batch, Local Runtime
    Affects Versions: 0.10.0
            Reporter: Maximilian Michels
            Assignee: Maximilian Michels
            Priority: Minor
             Fix For: 1.0.0, 0.10.1


The number of task slots for local execution are determined by the maximum parallelism found.
However, if a default parallelism has been set, this parallelism is used as the upper bound
for the number of task slots.

We should change this to always use the maximum parallelism as the number of task slots. Otherwise
jobs which include operators with a parallelism higher than the default parallelism fail to
execute locally.

For example, this fails

{noformat}
ExecutionEnvironment env = ..

env.setParallelism(2);

DataSet<Integer> set = env.fromElements(1,2,3,4)
    .map(el -> el+1)
    .setParallelism(4);

set.print();
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message