flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Fabian Hueske <fhue...@gmail.com>
Subject Re: Load distribution through the cluster
Date Tue, 19 Sep 2017 08:29:14 GMT
Hi,

Flink's scheduling aims to co-located tasks to reduce network communication
and ease the reasoning about resource/slot consumption.
A slot can execute one subtask of each operator of a program, i.e, a
parallel slice of the program.

You can control the scheduling of tasks by specifying resource groups. [1]
[2]

Best, Fabian

[1]
https://ci.apache.org/projects/flink/flink-docs-release-1.3/concepts/runtime.html#task-slots-and-resources
[2]
https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/datastream_api.html#task-chaining-and-resource-groups

2017-09-18 15:22 GMT+02:00 AndreaKinn <kinn6aer@hotmail.it>:

> Hi,
> I'm experimenting a bit with the cluster.
> I didn't set any options about sharing slots and chains hoping that Flink
> decided autonomously how to balance the load through the nodes of the
> cluster. My cluster is composed by one job and task manager and two task
> manager.
>
> I noted that every time I start the program, just one node is busy (at >
> 95%
> for each cpu core) while the other nodes are completely free (< 3%). Same
> arguments for the memory.
>
> So Flink doesn't balance the work on the nodes??
> I expected something like the cpu usage was distributed on every nodes.
>
>
>
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.
> n4.nabble.com/
>

Mime
View raw message