flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Aljoscha Krettek <aljos...@apache.org>
Subject Re: tasks running in parallel beyond configured parallelism/slots
Date Fri, 10 Feb 2017 16:05:53 GMT
Flink operators will not always (in fact almost never) run in a single
slot. Mostly the whole parallel sub-slice of a pipeline can run in one
slot, so in your case you get three parallel instances for every operator
in your topology and then one instance of each operator will sit in a slot.


On Thu, 9 Feb 2017 at 12:33 Antony Mayi <antonymayi@yahoo.com> wrote:

> Hi,
> I am new to Flink and bit confused about the execution pipeline of my
> Flink job. I run it on cluster of three task managers (flink 1.1.2) each
> configured with just single slot. I submit my job with parallelism set to 3.
> This is the global plan (low res - just to show the initial forking):
> http://pasteboard.co/weyMrFlZl.png
> This is a detail of the front part: http://pasteboard.co/wez3DVvfW.png
> My confusion is how comes all the parallel operations in the second column
> (10 operations) are being executed at the same time if there should be
> capacity for max of 3 running at once? Also they are all executed mostly on
> same node while the others are idle.
> Thanks for anything useful,
> Antony.

View raw message