flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai" <tzuli...@apache.org>
Subject Re: can flink do streaming from data sources other than Kafka?
Date Thu, 07 Sep 2017 08:04:11 GMT
Hi!


I am wondering if Flink can do streaming from data sources other than Kafka. For example can
Flink do streaming from a database like Cassandra, HBase, MongoDb to sinks like says Elastic
search or Kafka.

Yes, Flink currently supports various connectors for different sources and sinks. For an overview
you can check out this documentation [1]
Apache Bahir [2] also maintains some Flink connectors and is released separately.

Also for out of core stateful streaming. Is RocksDB the only option?
Currently, RocksDB is the only option for out-of-core state. There was some previous discussion
for a Cassandra state backend, though [3].

- Gordon

[1] https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/connectors/index.html
[2] http://bahir.apache.org/
[3] https://issues.apache.org/jira/browse/FLINK-4266

On 7 September 2017 at 2:58:38 PM, kant kodali (kanth909@gmail.com) wrote:

Hi All,

I am wondering if Flink can do streaming from data sources other than Kafka. For example can
Flink do streaming from a database like Cassandra, HBase, MongoDb to sinks like says Elastic
search or Kafka.

Also for out of core stateful streaming. Is RocksDB the only option? Can I use some other
key value store that has SQL interface (since RocksDB doesn't)?

Thanks,
kant
Mime
View raw message