flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Srinath Perera <hemap...@gmail.com>
Subject Minimal HA Setup for Apache Flink
Date Tue, 17 Oct 2017 09:53:20 GMT
Hi All,

I am trying to write an article comparing minimal HA(Highly available)
deployments of different streaming processing systems.

Basically, the question is if an organization has a limited workload, such
as 10k events per second, which might grow in the future, what is the
minimal setup they can use to run a highly available Stream Processor?

Could someone help answer following questions?

   1. How many nodes minimal Apache Flink HA setup needs? As I understood
   from [2], it is zookeeper nodes + 2 job managers without YARN and 1 job
   manager with YARN + worker nodes? Is this correct?
   2. As per [1], Zookeeper needs minimal 3 nodes to provide HA. Is there a
   way to run Apache Flink without HA?
   3. If someone runs Apache Flink without HA, but use state snapshots, how
   fast it can recover after a failure? ( ballpark figure)



Srinath Perera, Ph.D.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message