flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Flavio Pompermaier <pomperma...@okkam.it>
Subject Re: [DISCUSS] ElasticSearch in Flink Strategy
Date Mon, 09 Jan 2017 11:47:14 GMT
I think also https://issues.apache.org/jira/browse/FLINK-5353 should be
solved sooner or later...

Best,
Flavio

On 9 Jan 2017 12:24, "Tzu-Li (Gordon) Tai" <tzulitai@apache.org> wrote:

> Thanks for starting the discussion Robert.
>
> I agree that we should try to treat the Elasticsearch connectors in a
> similar fashion to the multiple Kafka connector versions.
>
> +1 for flink-connector-elasticsearch-base
> +1 to consolidating the documents
> +1 for keeping the ES connector in Flink
>
> I’d also like to proceed with this with https://github.com/apache/
> flink/pull/2767, by opening another new PR based on #2767 that also
> refactors the ES connectors & consolidates documentation.
> After that, we can proceed with https://github.com/apache/flink/pull/2861 for
> the proper at least once support across all ES versions.
>
> - Gordon
>
> On December 19, 2016 at 11:58:03 PM, Fabian Hueske (fhueske@gmail.com)
> wrote:
>
> Thanks for looking into this Robert.
> I agree, it would be good to share as much code as possible between the
> Elasticsearch connectors.
>
> +1 for flink-connector-elasticsearch-base
> +1 for keeping the connectors in Flink
>
> 2016-12-19 16:46 GMT+01:00 Robert Metzger <rmetzger@apache.org>:
>
> > Hi,
> > I'm wondering how we want to maintain the ElasticSearch sinks in Flink.
> >
> > Right now, we have two connectors, supporting the 1.x and 2.x versions.
> > There is a pending PR [1] for adding ES 5 support.
> >
> > All three connectors share a lot of code, and there is a pull request [2]
> > improving the ES 2.x connector. If we want to make the ES sinks have
> proper
> > at least once support, we need to do some more changes, and I would like
> to
> > do them once for all connectors.
> > I propose to create a new "flink-connector-elasticsearch-base" module
> with
> > the common code and then create modules for 1.x, 2.x and 5.x.
> > Also, I would merge the documentation pages into one.
> >
> > I would also be open to discuss whether we want to move the EL connectors
> > into Bahir.
> > For now, I'm against that, given that ES is among the more popular
> > connectors in Flink.
> >
> >
> > Regards,
> > Robert
> >
> > [1] https://github.com/apache/flink/pull/2767
> > [2] https://github.com/apache/flink/pull/2861
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message