flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai" <tzuli...@apache.org>
Subject Re: Regarding flink-cassandra-connectors
Date Tue, 26 Sep 2017 10:13:44 GMT
Ah, sorry I just realized Till also answered your question on your cross-post at dev@.
It’s usually fine to post questions to just a single mailing list :)


On 26 September 2017 at 12:10:55 PM, Tzu-Li (Gordon) Tai (tzulitai@apache.org) wrote:

Hi Jagadish,

Yes, you are right that the Flink Cassandra connector uses the Datastax drivers internally,
which is also the case for all the other Flink connectors; e.g., the Kafka connector uses
the Kafka Java client, Elasticearch connector uses the ES Java client, etc.

The main advantage when using these Flink first-class supported connectors is basically the
following:
- Most importantly, the connectors work with Flink’s checkpointing mechanism to achieve
exactly-once or at-least-once guarantees. You can read more about that here [1].
- The connectors are built on Flink’s abstractions of streaming sources / sinks. What this
means is you can basically swap out / plug-in / add sources or sinks to various external systems
without altering the main business logic in your processing pipeline. i.e., also sinking your
data to Elasticsearch would be as simple as also adding a Elasticsearch sink to your pipeline
output alongside your Cassandra sink.

Hope this clarifies some points for you!

Cheers,
Gordon

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.3/internals/stream_checkpointing.html

On 26 September 2017 at 11:03:16 AM, Jagadish Gangulli (jagadishg@gmail.com) wrote:

Hi,

I have been recently into the application development with flink. We are trying to use the
flink-apache connectors to get the data in and out from Cassandra.

We attempted both Datastax drivers and Flink-cassandra connectors.  In this process felt
that flink-cassandra connector is more of a wrapper on top of data stax cassandra drivers.

Hence could some one please explain the benefits of the flink-cassandra-connectors over the
data stax driver apis. We are looking for the APIs which are better in terms of performance.
Please let me know your thoughts.

Thanks & Regards,
Jagadisha G


Mime
View raw message