flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavio Pompermaier <pomperma...@okkam.it>
Subject Re: Flink slots, threads, task, etc
Date Tue, 11 Apr 2017 12:37:40 GMT
Any feedback here..?

On Wed, Apr 5, 2017 at 7:43 PM, Flavio Pompermaier <pompermaier@okkam.it>

> Hi to all,
> I had a very long but useful chat with Fabian and I understood a lot of
> concepts that was not clear at all to me. We started from the Flink runtime
> documentation page (https://ci.apache.org/projects/flink/flink-docs-
> release-1.2/concepts/runtime.html) but
> I discovered that the terminology is very inconsistent and misleading
> along the page...
> For example, one of the very first sentences is :
> "Flink chains operator subtasks together into tasks. Each task is executed
> by one thread."
> What I first understood was that every operator can be executed only by a
> single thread in all the cluster....probably it should be better "one
> thread per task slot" (at least).
> Moreover, if I'm not wrong, a Task Slot can execute only 1 subtask (aka
> parallel instance) of each task and there's no limit to the number of
> subtasks per slot (and this is not highlighted at all in that document).
> The only constraint is that they should belong to different tasks (right?).
> If there's a google doc version of that page I could try to rewrite it
> down in order to make it easier to understand some parts...however I still
> have some more questions:
>    1. Is it correct that a single Task Slot can execute only a single
>    subtask of each task and that this task is executed by a single thread
>    within the slot)?
>    2. If it so:
>       1. why at that page there's written "By default, Flink allows
>       subtasks to share slots even if they are subtasks of different tasks, so
>       long as they are from the same job"? It seems that it is more common to run
>       multiple subtasks of the same task (in a slot) than executing different
>       substasks of different tasks, although this is still permitted...from what
>       I understood a slot cannot run multiple subtask of the same task at all!
>       2. and why this constraint? Is there any good reason for that? A
>       subtask is mapped to 1 thread in the TaskManager, so why a TM with 2 slots
>       can run 2 subtasks of the same task (in the same JVM) while a TM with 1
>       slot cannot  (while it can execute an arbitrary number of subtasks of
>       different tasks)?
>    3. It it is not so, there's no images representing such a situation in
>    that page...
>    4. Isn't dangerous to allow (potentially) an unlimited number of
>    threads per TM slot??
> Cheers,
> Flavio

View raw message