flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From qi luo <luoqi...@gmail.com>
Subject Re: How many task managers can Flink efficiently scale to?
Date Sun, 11 Aug 2019 10:17:18 GMT
Hi Chad,

In our cases, 1~2k TMs with up to ~10k TM slots are used in one job. In general, the CPU/memory
of Job Manager should be increased with more TMs.

Regards,
Qi

> On Aug 11, 2019, at 2:03 AM, Chad Dombrova <chadrik@gmail.com> wrote:
> 
> Hi,
> I'm still on my task management investigation, and I'm curious to know how many task
managers people are reliably using with Flink.  We're currently using AWS | Thinkbox Deadline,
and we're able to easily utilize over 300 workers, and I've heard from other customers who
use several thousand, so I'm curious how Flink compares in this regard.  Also, what aspects
of the system begin to deteriorate at higher scales?
> 
> thanks in advance!
> 
> -chad
> 


Mime
View raw message