flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai" <tzuli...@apache.org>
Subject Re: [DISCUSS] ElasticSearch in Flink Strategy
Date Mon, 09 Jan 2017 11:24:26 GMT
Thanks for starting the discussion Robert.

I agree that we should try to treat the Elasticsearch connectors in a similar fashion to the
multiple Kafka connector versions.

+1 for flink-connector-elasticsearch-base
+1 to consolidating the documents
+1 for keeping the ES connector in Flink

I’d also like to proceed with this with https://github.com/apache/flink/pull/2767, by opening
another new PR based on #2767 that also refactors the ES connectors & consolidates documentation.
After that, we can proceed with https://github.com/apache/flink/pull/2861 for the proper
at least once support across all ES versions.

- Gordon

On December 19, 2016 at 11:58:03 PM, Fabian Hueske (fhueske@gmail.com) wrote:

Thanks for looking into this Robert.  
I agree, it would be good to share as much code as possible between the  
Elasticsearch connectors.  

+1 for flink-connector-elasticsearch-base  
+1 for keeping the connectors in Flink  

2016-12-19 16:46 GMT+01:00 Robert Metzger <rmetzger@apache.org>:  

> Hi,  
> I'm wondering how we want to maintain the ElasticSearch sinks in Flink.  
>  
> Right now, we have two connectors, supporting the 1.x and 2.x versions.  
> There is a pending PR [1] for adding ES 5 support.  
>  
> All three connectors share a lot of code, and there is a pull request [2]  
> improving the ES 2.x connector. If we want to make the ES sinks have proper  
> at least once support, we need to do some more changes, and I would like to  
> do them once for all connectors.  
> I propose to create a new "flink-connector-elasticsearch-base" module with  
> the common code and then create modules for 1.x, 2.x and 5.x.  
> Also, I would merge the documentation pages into one.  
>  
> I would also be open to discuss whether we want to move the EL connectors  
> into Bahir.  
> For now, I'm against that, given that ES is among the more popular  
> connectors in Flink.  
>  
>  
> Regards,  
> Robert  
>  
> [1] https://github.com/apache/flink/pull/2767  
> [2] https://github.com/apache/flink/pull/2861  
>  

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message