flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Maximilian Michels <...@apache.org>
Subject Re: What is the topic & offset used for in KeyedDeserializationSchema.deserialize()?
Date Tue, 19 Jan 2016 10:34:10 GMT
Hi Gordon,

You may use "topic" and "offset" for whatever you like. Note that this
is just an interface. If it does not work for your Kinesis adapter,
you may create a new interface. For existing usage of the
KeyedDeserializationSchema, please have a look at the
FlinkKafkaConsumer.

Cheers,
Max

On Tue, Jan 19, 2016 at 11:27 AM, Tzu-Li (Gordon) Tai
<tzulitai@gmail.com> wrote:
> Hi devs,
>
> I need a little help on clarification of what the arguments "topic" and
> "offset" is used for in KeyedDeserializationSchema.deserialize(). The main
> issue is that I'm currently in progress of implementing Flink Kinesis
> Consumer, and Kinesis offsets, unlike Kafka offsets which are incremental
> starting from 0, are digits that can only by stored in BigIntegers and
> generally doesn't increment by 1 between each data record.
>
> Just need to make sure that I won't be messing things up with these two
> values. A point to any part of the codebase where I can understand how Flink
> uses "topic" and "offset" in the deserialization schema would be perfect.
>
> Many thanks in advance!
>
> Cheers,
> Gordon
>
>
>
> --
> View this message in context: http://apache-flink-mailing-list-archive.1008284.n3.nabble.com/What-is-the-topic-offset-used-for-in-KeyedDeserializationSchema-deserialize-tp9911.html
> Sent from the Apache Flink Mailing List archive. mailing list archive at Nabble.com.

Mime
View raw message