flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dawid Wysakowicz <dwysakow...@apache.org>
Subject Re: Flink TaskManagers do not start until job is submitted in YARN
Date Mon, 24 Sep 2018 13:07:10 GMT
Hi Suraj,

As far as I know this was changed with FLIP-6 to allow dynamic resource
allocation.

Till, cced might know if there is a switch to restore old behavior or
are there plans to support it.

Best,

Dawid

On 24/09/18 12:24, suraj7 wrote:
> Hi,
>
> I am using Amazon EMR to run Flink Cluster on YARN. My setup consists of
> m4.large instances for 1 master and 2 core nodes. I have started the Flink
> Cluster on YARN with the command: flink-yarn-session -n 2 -d -tm 4096 -s 4.
>
> Flink Job Manager and Application Manager starts but there are no Task
> Managers running. The Flink Web interface shows 0 for task managers, task
> slots and slots available. However when I submit a job to flink cluster,
> then Task Managers get allocated and the job runs and the Web UI shows
> correct values as expected and goes back to 0 once the job is complete.
>
> I would like Task Managers to be running even when no Job is submitted as I
> want to use Flink's REST API to monitor and modify parallelism based on the
> available slots value while scaling Core Nodes. 
>
> Is there a configuration that I've missed which prevents Task Managers from
> running all the time?
>
> Thanks,
> Suraj
>
>
>
> --
> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/



Mime
View raw message