From user-return-23134-archive-asf-public=cust-asf.ponee.io@flink.apache.org Mon Sep 24 12:25:03 2018 Return-Path: X-Original-To: archive-asf-public@cust-asf.ponee.io Delivered-To: archive-asf-public@cust-asf.ponee.io Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by mx-eu-01.ponee.io (Postfix) with SMTP id EB8C7180649 for ; Mon, 24 Sep 2018 12:25:02 +0200 (CEST) Received: (qmail 51373 invoked by uid 500); 24 Sep 2018 10:25:01 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Delivered-To: mailing list user@flink.apache.org Received: (qmail 51363 invoked by uid 99); 24 Sep 2018 10:25:01 -0000 Received: from pnap-us-west-generic-nat.apache.org (HELO spamd2-us-west.apache.org) (209.188.14.142) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 24 Sep 2018 10:25:01 +0000 Received: from localhost (localhost [127.0.0.1]) by spamd2-us-west.apache.org (ASF Mail Server at spamd2-us-west.apache.org) with ESMTP id 4BFED1A258A for ; Mon, 24 Sep 2018 10:25:01 +0000 (UTC) X-Virus-Scanned: Debian amavisd-new at spamd2-us-west.apache.org X-Spam-Flag: NO X-Spam-Score: 2.284 X-Spam-Level: ** X-Spam-Status: No, score=2.284 tagged_above=-999 required=6.31 tests=[SPF_HELO_PASS=-0.001, SPF_SOFTFAIL=0.972, URI_HEX=1.313] autolearn=disabled Received: from mx1-lw-eu.apache.org ([10.40.0.8]) by localhost (spamd2-us-west.apache.org [10.40.0.9]) (amavisd-new, port 10024) with ESMTP id HQb-i44Zb6a2 for ; Mon, 24 Sep 2018 10:24:59 +0000 (UTC) Received: from n4.nabble.com (n4.nabble.com [199.38.86.66]) by mx1-lw-eu.apache.org (ASF Mail Server at mx1-lw-eu.apache.org) with ESMTP id F13905F2F0 for ; Mon, 24 Sep 2018 10:24:58 +0000 (UTC) Received: from n4.nabble.com (localhost [127.0.0.1]) by n4.nabble.com (Postfix) with ESMTP id 716F21C95B3D for ; Mon, 24 Sep 2018 05:24:58 -0500 (CDT) Date: Mon, 24 Sep 2018 05:24:58 -0500 (CDT) From: suraj7 To: user@flink.apache.org Message-ID: <1537784698462-0.post@n4.nabble.com> Subject: Flink TaskManagers do not start until job is submitted in YARN MIME-Version: 1.0 Content-Type: text/plain; charset=us-ascii Content-Transfer-Encoding: 7bit Hi, I am using Amazon EMR to run Flink Cluster on YARN. My setup consists of m4.large instances for 1 master and 2 core nodes. I have started the Flink Cluster on YARN with the command: flink-yarn-session -n 2 -d -tm 4096 -s 4. Flink Job Manager and Application Manager starts but there are no Task Managers running. The Flink Web interface shows 0 for task managers, task slots and slots available. However when I submit a job to flink cluster, then Task Managers get allocated and the job runs and the Web UI shows correct values as expected and goes back to 0 once the job is complete. I would like Task Managers to be running even when no Job is submitted as I want to use Flink's REST API to monitor and modify parallelism based on the available slots value while scaling Core Nodes. Is there a configuration that I've missed which prevents Task Managers from running all the time? Thanks, Suraj -- Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.nabble.com/