flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: [DISCUSS] ElasticSearch in Flink Strategy
Date Mon, 09 Jan 2017 13:53:26 GMT
Thanks for bringing this up again Flavio.

The good thing about a common base for all ES connectors is that we can fix
the issue for all ES versions (ideally).

Thanks a lot for taking care of my proposed changes Gordon!


On Mon, Jan 9, 2017 at 12:47 PM, Flavio Pompermaier <pompermaier@okkam.it>
wrote:

> I think also https://issues.apache.org/jira/browse/FLINK-5353 should be
> solved sooner or later...
>
> Best,
> Flavio
>
> On 9 Jan 2017 12:24, "Tzu-Li (Gordon) Tai" <tzulitai@apache.org> wrote:
>
> > Thanks for starting the discussion Robert.
> >
> > I agree that we should try to treat the Elasticsearch connectors in a
> > similar fashion to the multiple Kafka connector versions.
> >
> > +1 for flink-connector-elasticsearch-base
> > +1 to consolidating the documents
> > +1 for keeping the ES connector in Flink
> >
> > I’d also like to proceed with this with https://github.com/apache/
> > flink/pull/2767, by opening another new PR based on #2767 that also
> > refactors the ES connectors & consolidates documentation.
> > After that, we can proceed with https://github.com/apache/flin
> k/pull/2861 for
> > the proper at least once support across all ES versions.
> >
> > - Gordon
> >
> > On December 19, 2016 at 11:58:03 PM, Fabian Hueske (fhueske@gmail.com)
> > wrote:
> >
> > Thanks for looking into this Robert.
> > I agree, it would be good to share as much code as possible between the
> > Elasticsearch connectors.
> >
> > +1 for flink-connector-elasticsearch-base
> > +1 for keeping the connectors in Flink
> >
> > 2016-12-19 16:46 GMT+01:00 Robert Metzger <rmetzger@apache.org>:
> >
> > > Hi,
> > > I'm wondering how we want to maintain the ElasticSearch sinks in Flink.
> > >
> > > Right now, we have two connectors, supporting the 1.x and 2.x versions.
> > > There is a pending PR [1] for adding ES 5 support.
> > >
> > > All three connectors share a lot of code, and there is a pull request
> [2]
> > > improving the ES 2.x connector. If we want to make the ES sinks have
> > proper
> > > at least once support, we need to do some more changes, and I would
> like
> > to
> > > do them once for all connectors.
> > > I propose to create a new "flink-connector-elasticsearch-base" module
> > with
> > > the common code and then create modules for 1.x, 2.x and 5.x.
> > > Also, I would merge the documentation pages into one.
> > >
> > > I would also be open to discuss whether we want to move the EL
> connectors
> > > into Bahir.
> > > For now, I'm against that, given that ES is among the more popular
> > > connectors in Flink.
> > >
> > >
> > > Regards,
> > > Robert
> > >
> > > [1] https://github.com/apache/flink/pull/2767
> > > [2] https://github.com/apache/flink/pull/2861
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message