Return-Path: X-Original-To: apmail-flink-user-archive@minotaur.apache.org Delivered-To: apmail-flink-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id 446AA17D03 for ; Mon, 16 Mar 2015 07:27:35 +0000 (UTC) Received: (qmail 43273 invoked by uid 500); 16 Mar 2015 07:27:35 -0000 Delivered-To: apmail-flink-user-archive@flink.apache.org Received: (qmail 43202 invoked by uid 500); 16 Mar 2015 07:27:35 -0000 Mailing-List: contact user-help@flink.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@flink.apache.org Delivered-To: mailing list user@flink.apache.org Received: (qmail 43192 invoked by uid 99); 16 Mar 2015 07:27:35 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 16 Mar 2015 07:27:35 +0000 X-ASF-Spam-Status: No, hits=2.2 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_NONE,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of eleroy@msn.com designates 65.54.190.162 as permitted sender) Received: from [65.54.190.162] (HELO BAY004-OMC3S24.hotmail.com) (65.54.190.162) by apache.org (qpsmtpd/0.29) with ESMTP; Mon, 16 Mar 2015 07:27:09 +0000 Received: from BAY173-W40 ([65.54.190.188]) by BAY004-OMC3S24.hotmail.com over TLS secured channel with Microsoft SMTPSVC(7.5.7601.22751); Mon, 16 Mar 2015 00:27:07 -0700 X-TMN: [gruXGTUQzhDVk2vklYaq8w3rnzVXtUxe] X-Originating-Email: [eleroy@msn.com] Message-ID: Content-Type: multipart/alternative; boundary="_33407cba-ca8d-423d-b69e-33c5666fda41_" From: Emmanuel To: "user@flink.apache.org" Subject: Scaling a Flink cluster Date: Mon, 16 Mar 2015 07:27:07 +0000 Importance: Normal MIME-Version: 1.0 X-OriginalArrivalTime: 16 Mar 2015 07:27:07.0916 (UTC) FILETIME=[9BFA1CC0:01D05FBA] X-Virus-Checked: Checked by ClamAV on apache.org --_33407cba-ca8d-423d-b69e-33c5666fda41_ Content-Type: text/plain; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable Hello=2C In my understanding=2C the flink-conf.yaml is the one config file to config= ure a cluster.The slave file lists the slave nodes.they must both be on eve= ry node. I'm trying to understand what is the best strategy to scale a Flink cluster= since:- adding a node means adding an entry to the slave list and replicat= ing on every node. Does the cluster need to be restarted to take the new nodes into account? I= t seems like it.Having to replicate the file on all nodes is not super conv= enient. Restarting is even more trouble.Is there a way to scale a live clus= ter? If so how?Any link to the info would be helpful. Thanks = --_33407cba-ca8d-423d-b69e-33c5666fda41_ Content-Type: text/html; charset="iso-8859-1" Content-Transfer-Encoding: quoted-printable
Hello=2C

In m= y understanding=2C the flink-conf.yaml is the one config file to configure = a cluster.
The slave file lists the slave nodes.
they m= ust both be on every node.

I'm trying to understan= d what is the best strategy to scale a Flink cluster since:
- add= ing a node means adding an entry to the slave list and replicating on every= node.

Does the cluster need to be restarted to ta= ke the new nodes into account? It seems like it.
Having to replic= ate the file on all nodes is not super convenient. Restarting is even more = trouble.
Is there a way to scale a live cluster? If so how?
=
Any link to the info would be helpful.

Thanks=
= --_33407cba-ca8d-423d-b69e-33c5666fda41_--