flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Xintong Song <tonysong...@gmail.com>
Subject Re: How many task managers to launch for a job?
Date Fri, 24 May 2019 02:09:51 GMT
Hi black,

If you are running Flink on Yarn or Mesos, Flink will automatically
allocate resource and launch new TaskManagers as needed.

If you are using Flink standalone mode, then the easiest way is to enable
slot sharing and set all the vertices into the same group (which is by
default). In that way, the total slots (or number of TaskManagers if you
config on slot for each TaskManager) needed for running the job would be
the maximum parallelism of the job graph vertices. Further information on
slot sharing could be found here
<https://ci.apache.org/projects/flink/flink-docs-release-1.8/concepts/runtime.html#task-slots-and-resources>
.

Thank you~

Xintong Song



On Thu, May 23, 2019 at 11:49 PM black chase <chaseandblack@gmail.com>
wrote:

>
> Hi,
>
> I am redesigning the scheduler of the JobManager to place tasks of a job
> across TaskManagers accroding to a scheduling policy.
>
> I am reading the Flip-6 proposal and found that the common case is "one
> TaskManager launchs one slot", and "one Flink cluster serves one job". But
> I did not find how many TaskManagers to launch in a computing node. Is
> there any common practice for this ?
>
> --
> Best Regards!
> Pengcheng Duan
>

Mime
View raw message