flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai" <tzuli...@gmail.com>
Subject What is the topic & offset used for in KeyedDeserializationSchema.deserialize()?
Date Tue, 19 Jan 2016 10:27:56 GMT
Hi devs,

I need a little help on clarification of what the arguments "topic" and
"offset" is used for in KeyedDeserializationSchema.deserialize(). The main
issue is that I'm currently in progress of implementing Flink Kinesis
Consumer, and Kinesis offsets, unlike Kafka offsets which are incremental
starting from 0, are digits that can only by stored in BigIntegers and
generally doesn't increment by 1 between each data record.

Just need to make sure that I won't be messing things up with these two
values. A point to any part of the codebase where I can understand how Flink
uses "topic" and "offset" in the deserialization schema would be perfect.

Many thanks in advance!


View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/What-is-the-topic-offset-used-for-in-KeyedDeserializationSchema-deserialize-tp9911.html
Sent from the Apache Flink Mailing List archive. mailing list archive at Nabble.com.

View raw message