flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: How to understand slot?
Date Tue, 18 Aug 2015 08:22:48 GMT
Hi!

There is a little bit of documentation about the scheduling and the slots

  - In the config reference:
https://ci.apache.org/projects/flink/flink-docs-master/setup/config.html#configuring-taskmanager-processing-slots

  - In the internals docs:
https://ci.apache.org/projects/flink/flink-docs-master/internals/job_scheduling.html


For <our other questions, here are some brief answers:

1) Shared slots are very useful for pipelined execution. Shared slots mean
that (in the example of MapReduce), one slot can hold one mapper and one
reducer. Mappers and reducers run at the same time when using pipelined
execution.

2) A good choice for the number of slots is the number of CPU cores in the
processor.

3) For sources, Flink picks a random TaskManager (splits are then assigned
locality aware to the sources). For all tasks after sources, Flink tries to
co-locate them with their input(s), unless they have so many inputs that
co-location makes no difference (each parallel reducer task has all mapper
tasks as inputs).


Greetings,
Stephan



On Tue, Aug 18, 2015 at 9:29 AM, Zhangrucong <zhangrucong@huawei.com> wrote:

> When I read the schedule code in job manager. I have flowing questions:
>
>
>
> 1、  How to decide a job vertex to deploy in a shared slot?  What is the
> benefit deploy vertexes in a shared slot?
>
> 2、  How to decide a task manager has how many slots?
>
> 3、  If there are many task managers, when allocate a new slot, how to
> decide to use which slot in which task manger?
>
> 4、  If there have detail documents about schedule?
>
>
>
> Thank you for any suggestions in advance!
>
>
>
>
>

Mime
View raw message