flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Srinath Perera <hemap...@gmail.com>
Subject Minimal HA Setup for Apache Flink
Date Tue, 17 Oct 2017 09:53:20 GMT
Hi All,

I am trying to write an article comparing minimal HA(Highly available)
deployments of different streaming processing systems.

Basically, the question is if an organization has a limited workload, such
as 10k events per second, which might grow in the future, what is the
minimal setup they can use to run a highly available Stream Processor?

Could someone help answer following questions?

   1. How many nodes minimal Apache Flink HA setup needs? As I understood
   from [2], it is zookeeper nodes + 2 job managers without YARN and 1 job
   manager with YARN + worker nodes? Is this correct?
   2. As per [1], Zookeeper needs minimal 3 nodes to provide HA. Is there a
   way to run Apache Flink without HA?
   3. If someone runs Apache Flink without HA, but use state snapshots, how
   fast it can recover after a failure? ( ballpark figure)

Thanks
Srinath


   1.
   https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-hdfs/HDFSHighAvailabilityWithNFS.html#Deploying_ZooKeeper
   2.
   https://ci.apache.org/projects/flink/flink-docs-release-1.3/setup/jobmanager_high_availability.html#standalone-cluster-high-availability


-- 
============================
Srinath Perera, Ph.D.
   http://people.apache.org/~hemapani/
   http://srinathsview.blogspot.com/

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message