flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Márton Balassi <balassi.mar...@gmail.com>
Subject Re: Apache Flink and Kudu integration
Date Thu, 10 Nov 2016 09:30:58 GMT
Hi Ruben,

Thanks. Let us know how you progress. Funny enough today I am playing with
the Spark Kudu connector, I would be interested in checking out your code
as soon as you have something tangible.

Best,

Marton

On Thu, Nov 10, 2016 at 10:19 AM, Fabian Hueske <fhueske@gmail.com> wrote:

> Hi Ruben,
>
> that sounds great!
> In case you are planning to contribute your connector, you should have a
> look at Apache Bahir [1].
> Bahir is a project that collects connectors and other extensions of
> distributed analytics platforms (currently Flink and Spark).
>
> As of now, it offers Flink connectors to Redis, Flume, and ActiveMQ.
>
> Cheers,
> Fabian
>
> [1] http://bahir.apache.org/
>
>
>
> 2016-11-10 8:56 GMT+01:00 <ruben.casado.tejedor@accenture.com>:
>
> > Hi,
> >
> > I am starting a PoC to do that. I will try to develop both source and
> > sink. I will let you know asap I have something ;-)
> >
> > Best
> >
> > -----Original Message-----
> > From: Márton Balassi [mailto:balassi.marton@gmail.com]
> > Sent: viernes, 28 de octubre de 2016 8:50
> > To: dev@flink.apache.org
> > Subject: Re: Apache Flink and Kudu integration
> >
> > Hi Ruben,
> >
> > I am currently not aware of such an effort, but I definitely do agree
> that
> > it is an interesting pattern to investigate. As a motivation you could
> have
> > a look at the Spark connector implementations to see the Kudu APIs in
> use.
> > For that I would recommend the DataSource API implementation that is now
> > part of Spark or Ted Malaska's prototype [2] that is bit less complex
> thus
> > might be easier to read.
> >
> > Let us know if you decide to give the implementation a try.
> >
> > [1]
> > https://urldefense.proofpoint.com/v2/url?u=https-3A__kudu.
> > apache.org_docs_developing.html-23-5Fkudu-5Fintegration-
> > 5Fwith-5Fspark&d=DQIFaQ&c=eIGjsITfXP_y-DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=
> > brkRAgrW3LbdVDOiRLzI7SFUIWBL5aa2MIfENljA8xoe0lFg2u3-S6GnFTH7Pbmc&m=
> > ncxdkfO9tu7KuKTt6QEv5kC5nYPTuBj2fFI65S3vg3g&s=ut78uIYh-IHXL-
> > ZV2ChjskV3u9ChY3UE1Vkso5z7M-c&e=
> > [2] https://urldefense.proofpoint.com/v2/url?u=https-3A__github.
> > com_tmalaska_SparkOnKudu&d=DQIFaQ&c=eIGjsITfXP_y-
> > DLLX0uEHXJvU8nOHrUK8IrwNKOtkVU&r=brkRAgrW3LbdVDOiRLzI7SFUIWBL5a
> > a2MIfENljA8xoe0lFg2u3-S6GnFTH7Pbmc&m=ncxdkfO9tu7KuKTt6QEv5kC5nYPTuB
> > j2fFI65S3vg3g&s=rx8u6LdU2OCmU1lCD-_8o_9R3naC5viVIO-ijINS2aI&e=
> >
> > Best,
> >
> > Marton
> >
> > On Fri, Oct 28, 2016 at 8:33 AM, <ruben.casado.tejedor@accenture.com>
> > wrote:
> >
> > > Hi all,
> > >
> > > Is there any PoC about reading/writing from/to Kudu? I think the flow
> > > kafka-flink-kudu is an interesting pattern. I would like to evaluate
> > > it so please let me know if there is any existing attempt to avoid
> > > starting from scratch. Advices are welcomed :)
> > >
> > > Best
> > >
> > >
> > > ----------------------------------------
> > > Rubén Casado Tejedor, PhD
> > > > accenture digital
> > > Big Data Manager
> > > ':+ 34 629 009 429
> > > *:ruben.casado.tejedor@accenture.com
> > >
> > >
> > > ________________________________
> > >
> > > This message is for the designated recipient only and may contain
> > > privileged, proprietary, or otherwise confidential information. If you
> > > have received it in error, please notify the sender immediately and
> > > delete the original. Any other use of the e-mail by you is prohibited.
> > > Where allowed by local law, electronic communications with Accenture
> > > and its affiliates, including e-mail and instant messaging (including
> > > content), may be scanned by our systems for the purposes of
> > > information security and assessment of internal compliance with
> > Accenture policy.
> > > ____________________________________________________________
> > > __________________________
> > >
> > > www.accenture.com
> > >
> >
> > ________________________________
> >
> > This message is for the designated recipient only and may contain
> > privileged, proprietary, or otherwise confidential information. If you
> have
> > received it in error, please notify the sender immediately and delete the
> > original. Any other use of the e-mail by you is prohibited. Where allowed
> > by local law, electronic communications with Accenture and its
> affiliates,
> > including e-mail and instant messaging (including content), may be
> scanned
> > by our systems for the purposes of information security and assessment of
> > internal compliance with Accenture policy.
> > ____________________________________________________________
> > __________________________
> >
> > www.accenture.com
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message