flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Gary Yao <g...@ververica.com>
Subject Re: Localenvironment jobcluster ha High availability
Date Wed, 11 Dec 2019 12:24:28 GMT
Hi Eric,

What you say should be possible because your job will be executed in a
MiniCluster [1] which has HA support. I have not tried this out myself,
and I am not aware that people are doing this in production. However,
there are integration tests that use MiniCluster + ZooKeeper [2].

Best,
Gary

[1]
https://github.com/apache/flink/blob/eddad99123525211c900102206384dacaf8385fc/flink-runtime/src/main/java/org/apache/flink/runtime/minicluster/MiniCluster.java
[2]
https://github.com/apache/flink/blob/eddad99123525211c900102206384dacaf8385fc/flink-tests/src/test/java/org/apache/flink/test/checkpointing/EventTimeWindowCheckpointingITCase.java

On Tue, Dec 3, 2019 at 7:21 PM Eric HOFFMANN <
eric.hoffmann.e@thalesdigital.io> wrote:

> Hi, i use a jobcluster (1 manager and 1 worker) in kubernetes for
> streaming application, i would like to have the lightest possible solution,
> is it possible to use a localenvironment (manager and worker embeded) and
> still have HA with zookeeper in this mode?, I mean kubernetes will restart
> the job, in the case of jobcluster, metadata are retrieve from zookeeper
> and data from S3 or hdfs, is this pattern the same in localenvironment ?
> Thx
> Eric
>
> This message contains confidential information and is intended only for
> the individual(s) addressed in the message. If you are not the named
> addressee, you should not disseminate, distribute, or copy this e-mail. If
> you are not the intended recipient, you are notified that disclosing,
> distributing, or copying this e-mail is strictly prohibited.
>

Mime
View raw message