Mailing-List: contact user-help@flink.apache.org; run by ezmlm
Precedence: bulk
Date: Tue, 26 Sep 2017 12:13:44 +0200
From: "Tzu-Li (Gordon) Tai" <tzulitai@apache.org>
To: Jagadish Gangulli <jagadishg@gmail.com>, user@flink.apache.org
Cc: Till Rohrmann <trohrmann@apache.org>
Message-ID: <etPan.59ca2858.6b27ca62.118@apache.org>
In-Reply-To: <etPan.59ca27ab.29869167.118@apache.org>
References: <CAGga2Erc0RPtsWwxu6g164y1SAr0aUdAZ=hB8XLGttJuT1YKXQ@mail.gmail.com>
 <etPan.59ca27ab.29869167.118@apache.org>
Subject: Re: Regarding flink-cassandra-connectors
MIME-Version: 1.0
Content-Type: multipart/alternative; boundary="59ca2858_687c4373_118"
archived-at: Tue, 26 Sep 2017 10:13:50 -0000

--59ca2858_687c4373_118
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

Ah, sorry I just realized Till also answered your question on your cross-=
post at dev=40.
It=E2=80=99s usually fine to post questions to just a single mailing list=
 :)


On 26 September 2017 at 12:10:55 PM, Tzu-Li (Gordon) Tai (tzulitai=40apac=
he.org) wrote:

Hi Jagadish,

Yes, you are right that the =46link Cassandra connector uses the Datastax=
 drivers internally, which is also the case for all the other =46link con=
nectors; e.g., the Kafka connector uses the Kafka Java client, Elasticear=
ch connector uses the ES Java client, etc.

The main advantage when using these =46link first-class supported connect=
ors is basically the following:
- Most importantly, the connectors work with =46link=E2=80=99s checkpoint=
ing mechanism to achieve exactly-once or at-least-once guarantees. You ca=
n read more about that here =5B1=5D.
- The connectors are built on =46link=E2=80=99s abstractions of streaming=
 sources / sinks. What this means is you can basically swap out / plug-in=
 / add sources or sinks to various external systems without altering the =
main business logic in your processing pipeline. i.e., also sinking your =
data to Elasticsearch would be as simple as also adding a Elasticsearch s=
ink to your pipeline output alongside your Cassandra sink.

Hope this clarifies some points for you=21

Cheers,
Gordon

=5B1=5D=C2=A0https://ci.apache.org/projects/flink/flink-docs-release-1.3/=
internals/stream=5Fcheckpointing.html

On 26 September 2017 at 11:03:16 AM, Jagadish Gangulli (jagadishg=40gmail=
.com) wrote:

Hi,

I have been recently into the application development with flink. We are =
trying to use the flink-apache connectors to get the data in and out from=
 Cassandra.

We attempted both Datastax drivers and =46link-cassandra connectors.=C2=A0=
 In this process felt that flink-cassandra connector is more of a wrapper=
 on top of data stax cassandra drivers.

Hence could some one please explain the benefits of the flink-cassandra-c=
onnectors over the data stax driver apis. We are looking for the APIs whi=
ch are better in terms of performance. Please let me know your thoughts.

Thanks & Regards,
Jagadisha G


--59ca2858_687c4373_118
Content-Type: text/html; charset="utf-8"
Content-Transfer-Encoding: quoted-printable
Content-Disposition: inline

<html><head><style>body=7Bfont-family:Helvetica,Arial;font-size:13px=7D</=
style></head><body style=3D=22word-wrap: break-word; -webkit-nbsp-mode: s=
pace; -webkit-line-break: after-white-space;=22><div id=3D=22bloop=5Fcust=
omfont=22 style=3D=22font-family:Helvetica,Arial;font-size:13px; color: r=
gba(0,0,0,1.0); margin: 0px; line-height: auto;=22>Ah, sorry I just reali=
zed Till also answered your question on your cross-post at dev=40.</div><=
div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial;=
font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22=
>It=E2=80=99s usually fine to post questions to just a single mailing lis=
t :)</div> <br> <div class=3D=22bloop=5Fsign=22 id=3D=22bloop=5Fsign=5F15=
06420707504622848=22></div> <br><p class=3D=22airmail=5Fon=22>On 26 Septe=
mber 2017 at 12:10:55 PM, Tzu-Li (Gordon) Tai (<a href=3D=22mailto:tzulit=
ai=40apache.org=22>tzulitai=40apache.org</a>) wrote:</p> <blockquote type=
=3D=22cite=22 class=3D=22clean=5Fbq=22><span><div style=3D=22word-wrap: b=
reak-word; -webkit-nbsp-mode: space; -webkit-line-break: after-white-spac=
e;=22><div></div><div>


<title></title>


<div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial=
;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22=
>
Hi Jagadish,</div>
<div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial=
;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22=
>
<br></div>
<div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial=
;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22=
>
Yes, you are right that the =46link Cassandra connector uses the
Datastax drivers internally, which is also the case for all the
other =46link connectors; e.g., the Kafka connector uses the Kafka
Java client, Elasticearch connector uses the ES Java client,
etc.</div>
<div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial=
;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22=
>
<br></div>
<div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial=
;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22=
>
The main advantage when using these =46link first-class supported
connectors is basically the following:</div>
<div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial=
;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22=
>
- Most importantly, the connectors work with =46link=E2=80=99s checkpoint=
ing
mechanism to achieve exactly-once or at-least-once guarantees. You
can read more about that here =5B1=5D.</div>
<div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial=
;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22=
>
- The connectors are built on =46link=E2=80=99s abstractions of streaming=

sources / sinks. What this means is you can basically swap out /
plug-in / add sources or sinks to various external systems without
altering the main business logic in your processing pipeline. i.e.,
also sinking your data to Elasticsearch would be as simple as also
adding a Elasticsearch sink to your pipeline output alongside your
Cassandra sink.</div>
<div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial=
;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22=
>
<br></div>
<div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial=
;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22=
>
Hope this clarifies some points for you=21</div>
<div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial=
;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22=
>
<br></div>
<div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial=
;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22=
>
Cheers,</div>
<div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial=
;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22=
>
Gordon</div>
<div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial=
;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22=
>
<br></div>
<div id=3D=22bloop=5Fcustomfont=22 style=3D=22font-family:Helvetica,Arial=
;font-size:13px; color: rgba(0,0,0,1.0); margin: 0px; line-height: auto;=22=
>
=5B1=5D&nbsp;https://ci.apache.org/projects/flink/flink-docs-release-1.3/=
internals/stream=5Fcheckpointing.html</div>
<div class=3D=22bloop=5Fsign=22 id=3D=22bloop=5Fsign=5F150642013917176115=
2=22></div>
<br>
<p class=3D=22airmail=5Fon=22>On 26 September 2017 at 11:03:16 AM, Jagadi=
sh
Gangulli (<a href=3D=22mailto:jagadishg=40gmail.com=22>jagadishg=40gmail.=
com</a>) wrote:</p>
<blockquote type=3D=22cite=22 class=3D=22clean=5Fbq=22>
<div>
<div>
<div dir=3D=22ltr=22>
<div class=3D=22gmail=5Fquote=22>
<div dir=3D=22ltr=22>
<div>
<div>
<div><span>Hi,<br></span></div>
<div><span><br></span></div>
<div><span>I have been recently into the application development
with flink. We are trying to use the flink-apache connectors to get
the data in and out from Cassandra.<br></span></div>
<div><span><br></span></div>
<div><span>We attempted both Datastax drivers and =46link-cassandra
connectors.&nbsp; In this process felt that flink-cassandra
connector is more of a wrapper on top of data stax cassandra
drivers.</span></div>
<div><span><br></span></div>
<div><span>Hence could some one please explain the benefits of the
flink-cassandra-connectors over the data stax driver apis. We are
looking for the APIs which are better in terms of performance.
Please let me know your thoughts.<br></span></div>
<span><br></span></div>
<span>Thanks &amp; Regards,<br></span></div>
<span>Jagadisha G</span></div>
</div>
<span><br></span></div>
</div>
</div>
</blockquote>


</div></div></span></blockquote></body></html>
--59ca2858_687c4373_118--