flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From suraj7 <su...@amagi.com>
Subject Flink TaskManagers do not start until job is submitted in YARN
Date Mon, 24 Sep 2018 10:24:58 GMT

I am using Amazon EMR to run Flink Cluster on YARN. My setup consists of
m4.large instances for 1 master and 2 core nodes. I have started the Flink
Cluster on YARN with the command: flink-yarn-session -n 2 -d -tm 4096 -s 4.

Flink Job Manager and Application Manager starts but there are no Task
Managers running. The Flink Web interface shows 0 for task managers, task
slots and slots available. However when I submit a job to flink cluster,
then Task Managers get allocated and the job runs and the Web UI shows
correct values as expected and goes back to 0 once the job is complete.

I would like Task Managers to be running even when no Job is submitted as I
want to use Flink's REST API to monitor and modify parallelism based on the
available slots value while scaling Core Nodes. 

Is there a configuration that I've missed which prevents Task Managers from
running all the time?


Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/

View raw message