flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Shannon Carey <sca...@expedia.com>
Subject Re: Flink HA with Kubernetes, without Zookeeper
Date Mon, 21 Aug 2017 14:15:47 GMT
Zookeeper should still be necessary even in that case, because it is where the JobManager stores
information which needs to be recovered after the JobManager fails.

We're eyeing https://github.com/coreos/zetcd as a way to run Zookeeper on top of Kubernetes'
etcd cluster so that we don't have to rely on a separate Zookeeper cluster. However, we haven't
tried it yet.

-Shannon

From: Hao Sun <hasun@zendesk.com<mailto:hasun@zendesk.com>>
Date: Sunday, August 20, 2017 at 9:04 PM
To: "user@flink.apache.org<mailto:user@flink.apache.org>" <user@flink.apache.org<mailto:user@flink.apache.org>>
Subject: Flink HA with Kubernetes, without Zookeeper

Hi, I am new to Flink and trying to bring up a Flink cluster on top of Kubernetes.

For HA setup, with kubernetes, I think I just need one job manager and do not need Zookeeper?
I will store all states to S3 buckets. So in case of failure, kubernetes can just bring up
a new job manager without losing anything?

I want to confirm my assumptions above make sense. Thanks
Mime
View raw message