flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Timo Walther <twal...@apache.org>
Subject Re: Sample project on real time ingestion of more than 1Billion events at a time
Date Mon, 23 Oct 2017 15:01:38 GMT
Hi Deepak,

actually, every Flink example program can scale up to millions of events 
and more. The Flink APIs are designed to abstract the business logic 
from the parallelism. You just need to implement the interfaces that 
Flink provides.

If you are interesting in some example program, I can recommend this 
blog post [1]. A typical storage after processing, can be Kafka, 
EleasticSearch or you skip the additional system and use Flink's 
queryable state feature [2].

Hope that helps.


[1] https://data-artisans.com/blog/extending-the-yahoo-streaming-benchmark
[2] https://data-artisans.com/blog/queryable-state-use-case-demo

Am 10/23/17 um 4:42 PM schrieb Deepak Sharma:
> Hi All
> Can anyone point me to sample code that can scale upto real time 
> ingestion of events in millions or billion ?
> Also i need reference to which analytic engine to store the 
> transformed events for real time querying .
> Thanks a lot in advance.
> --Deepak

View raw message