flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Kostas Kloudas <k.klou...@data-artisans.com>
Subject Re: When using Flink for CEP, can the data in Cassandra database be used for state
Date Thu, 09 Nov 2017 14:01:17 GMT
Hi Shyla,

Happy to hear that you are experimenting with CEP!

For enriching your input stream with data from Cassandra (or whichever external storage system)
you could use:
* either the AsyncIO functionality offered by Flink (https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/stream/asyncio.html
<https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/stream/asyncio.html>)
* or, iff all your database fits in memory, you could write a ProcessFunction (https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/stream/process_function.html
<https://ci.apache.org/projects/flink/flink-docs-release-1.3/dev/stream/process_function.html>)
which loads the database in memory in the open() method, and then uses the data accordingly.

Afterwards, you can use the resulting (enriched) DataStream to feed it into CEP for further
processing.

Hope this helps!
Kostas

> On Nov 9, 2017, at 12:08 AM, shyla deshpande <deshpandeshyla@gmail.com> wrote:
> 
> Hello all,
> 
> I am new to Flink.
> 
> We have our data in Cassandra database. We have a use case for CEP. 
> I am checking out if Flink fits well for us.  
> 
> When processing the event data, I may want to pull data for the cassandra database like
the user profile and join with the event data.
> 
> Is there a way to do that?  I appreciate your help. 
> 
> Thanks


Mime
View raw message