flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Pei HE <...@apache.org>
Subject Re: What is the recommended way to run flink with high availability on AWS?
Date Wed, 21 Aug 2019 04:22:15 GMT
Thanks Kali for the information. However, it doesn't work for me, because I
need features in Flink 1.7.x or later and use manged Amazon MSK.
--
Pei



On Tue, Aug 20, 2019 at 7:17 PM sri hari kali charan Tummala <
kali.tummala@gmail.com> wrote:

> check this out AWS managed Flink cluster in aws kinesis, in AWS kinesis
> try to create Kinesis data analytics application using flink instead of sql.
>
> On Tue, Aug 20, 2019 at 9:28 PM Pei HE <pei@apache.org> wrote:
>
>> Hi there,
>> Looking at Flink document [1], EMR is the recommended way to run Flink on
>> AWS. However, Flink is currently not supported in an EMR cluster with
>> multiple master nodes [2].
>>
>> For example, I can create a EMR HA cluster with Zookeeper, Hive, etc.
>> But, if I add Flink to the application list when I create the cluster, I
>> got the following error from aws cli:
>> "An error occurred (ValidationException) when calling the RunJobFlow
>> operation: HA clusters do not support the specified applications: Flink.
>> Revise the applications and resubmit.".
>>
>> My question is what is the recommended way to run flink with high
>> availability on AWS.
>>
>> Thanks
>> --
>> Pei
>>
>> [1]:
>> https://ci.apache.org/projects/flink/flink-docs-stable/ops/deployment/aws.html
>> [2]:
>> https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-plan-ha-applications.html
>>
>>
>
> --
> Thanks & Regards
> Sri Tummala
>
>

Mime
View raw message