flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jia Teoh <jiat...@gmail.com>
Subject Kafka 0.10.x event time with windowing
Date Tue, 16 May 2017 20:44:54 GMT
Hi,

I'm trying to use KafkaConsumer010 as a source for a windowing job on event
time, as provided by Kafka. According to the kafka connector doc (link
<https://ci.apache.org/projects/flink/flink-docs-release-1.2/dev/connectors/kafka.html#using-kafka-timestamps-and-flink-event-time-in-kafka-010>),
I've set the time characteristic to event time (
streamExecutionEnvironment.setStreamTimeCharacteristic(TimeCharacteristic.EventTime))
and am using KafkaConsumer010 along with Kafka 0.10.2. I've also set up
windowing:
"stream.windowAll(TumblingEventTimeWindows.of(Time.milliseconds(windowMillis)))"
(using timeWindowAll appears to be equivalent as well).

Using these configurations I can verify that data is read from Kafka.
However, the event time windows never trigger even when data is loaded for
much longer than the window size. Is there an additional configuration I am
missing?

I have verified that the Kafka messages have timestamps. The docs mention
that there is no need for a timestamp extractor, but using one to
explicitly assign the current time does result in windows being triggered.

Thanks,
Jia Teoh

Mime
View raw message