flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yu Yang <yuyan...@gmail.com>
Subject read a finite number of messages from Kafka using Kafka connector without extending it?
Date Sat, 16 Feb 2019 00:10:35 GMT
Hi,

We are considering to use Flink SQL for ad hoc data analytics on real-time
Kafka data, and want to limit the queries to process data in the past 5-10
minutes. To achieve that, one possible approach is to extend the current
Kafka connect to have it only read messages in a given period of time to
generate a finite DataStream. I am wondering if there is an alternative to
this approach. Any suggestions will be very much appreciated.

Regards,
-Yu

Mime
View raw message