Mailing-List: contact dev-help@flink.apache.org; run by ezmlm
Precedence: bulk
Reply-To: dev@flink.apache.org
MIME-Version: 1.0
From: Srinath Perera <hemapani@gmail.com>
Date: Tue, 17 Oct 2017 15:23:20 +0530
Message-ID: <CAMtVhuUZS6WqvfWgRyoob+sqHZNgqHuS0=-tkyhZi2n9JZyo5g@mail.gmail.com>
Subject: Minimal HA Setup for Apache Flink
To: dev@flink.apache.org
Content-Type: multipart/alternative; boundary="94eb2c184b14db95d8055bbb171c"
archived-at: Tue, 17 Oct 2017 09:53:26 -0000

--94eb2c184b14db95d8055bbb171c
Content-Type: text/plain; charset="UTF-8"

Hi All,

I am trying to write an article comparing minimal HA(Highly available)
deployments of different streaming processing systems.

Basically, the question is if an organization has a limited workload, such
as 10k events per second, which might grow in the future, what is the
minimal setup they can use to run a highly available Stream Processor?

Could someone help answer following questions?

   1. How many nodes minimal Apache Flink HA setup needs? As I understood
   from [2], it is zookeeper nodes + 2 job managers without YARN and 1 job
   manager with YARN + worker nodes? Is this correct?
   2. As per [1], Zookeeper needs minimal 3 nodes to provide HA. Is there a
   way to run Apache Flink without HA?
   3. If someone runs Apache Flink without HA, but use state snapshots, how
   fast it can recover after a failure? ( ballpark figure)

Thanks
Srinath


   1.
   https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithNFS.html#Deploying_ZooKeeper
   2.
   https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/jobmanager_high_availability.html#standalone-cluster-high-availability


-- 
============================
Srinath Perera, Ph.D.
   http://people.apache.org/~hemapani/
   http://srinathsview.blogspot.com/

--94eb2c184b14db95d8055bbb171c--