flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ufuk Celebi <...@apache.org>
Subject Re: Task Parallelism in a Cluster
Date Tue, 01 Dec 2015 14:44:20 GMT

> On 01 Dec 2015, at 15:26, Kashmar, Ali <Ali.Kashmar@emc.com> wrote:
> Is there a way to make a task cluster-parallelizable? I.e. Make sure the
> parallel instances of the task are distributed across the cluster. When I
> run my flink job with a parallelism of 16, all the parallel tasks are
> assigned to the first task manager.

No, currently this is not possible*. But it can actually be beneficial for job performance
if everything is running locally on a single task manager. Is there a specific reason that
you want to spread it out?

– Ufuk

* There was a pull request by Till Rohrmann, which never made it into the main code: https://github.com/apache/flink/pull/60

View raw message