flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Tony Wei <tony19920...@gmail.com>
Subject Re: Balancing operation across machines
Date Sun, 22 Oct 2017 03:28:25 GMT

I think this patch[1] is what you need and it might be merged in 1.4.0
AFAIK, there is no way to change the behavior of scheduler from public API.
One work around that I could come up with is to give each TM one available
slot, make all operators in the same SlotSharingGroup and set the
parallelism as the number of TMs.
Hope this will help you.

Tony Wei

[1] https://issues.apache.org/jira/browse/FLINK-7851

2017-10-22 10:35 GMT+08:00 Le Xu <sharonxu65@gmail.com>:

> Hello!
> Just wondering if there is a way to load-balancing all executors on one
> operator across the machine. I have a reduce operator that has less
> parallelism level than the total number of slots in the cluster. Currently
> flink tries to compact executors in an operator and schedule them to some
> of the machines in the cluster. Just wondering if there is a way to make
> Flink schedule these executor evenly across the machines (e.g. one executor
> per machine.)?
> Thanks!
> Le

View raw message