flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephen <bahuash...@gmail.com>
Subject Re: How to deploy Flink in a geo-distributed environment
Date Fri, 29 Jun 2018 15:53:28 GMT

On Thu, Jun 28, 2018 at 7:32 PM Tzu-Li (Gordon) Tai <tzulitai@apache.org>

> Hi,
> It should be possible to deploy a single Flink cluster across
> geo-distributed nodes, but Flink currently offers no optimization for such
> a specific use case.
> AFAIK, the general pattern for dealing with geographically distributed
> data sources right now, would be to replicate data across clusters, such
> that they end up within a same target central destination before a
> processing framework such as Flink handles them.
> Good designs for multi-cluster data replication across, say multiple Kafka
> cluster, would be out of scope of this mailing list, though.
> Some quick googling led me to slides such as [1], but I'm sure there's
> will be more resources out there.
> Cheers,
> Gordon
> [1]
> https://www.slideshare.net/ConfluentInc/common-patterns-of-multi-datacenter-architectures-with-apache-kafka
> On Wed, Jun 27, 2018 at 3:08 AM Stephen <bahuasheng@gmail.com> wrote:
>> Hi,
>> Can Flink be deployed in a geo-distributed environment instead of being
>> in local clusters?
>> As far as I know, raw data should be moved to local cloud environment or
>> local clusters before Flink handle it. Consider this situation where data
>> sources are on different areas which might be cross different countries
>> that moving data with wlan is slow and expensive. How to solve this
>> problem? Is there solution for this now?
>> Thanks.

View raw message