flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: Flink Kafka from Time Offset
Date Mon, 30 May 2016 12:28:09 GMT
Hi Simon,

Timestamp-awareness has been added to Kafka 0.10 only [1]. I'm not sure if
the 0.9 connector code of Flink will support Kafka 0.10 immediately.
Another way of handling the issue would be
- Implementing a custom offset-metadata system (storing the timestamp for
some of the offsets (say every minute or so))
- Implement this feature into Flink's Kafka connector:
https://issues.apache.org/jira/browse/FLINK-3123

Regarding your second question: What "id"s do you mean?
If you are not using savepoints, the checkpoint state will not be
automatically restored.

Regards,
Robert

[1] https://issues.apache.org/jira/browse/KAFKA-2511


On Mon, May 30, 2016 at 1:45 PM, simon peyer <simon.peyer@soom-it.ch> wrote:

> Hi together
>
> I'm using flink 1.0.1 and a FlinkKafkaConsumer09.
> I'm very interested in getting data from a specific Time offset in Kafka.
>
> Is there a property which can do this?
> Or is there another way of handling such issues?
>
> I'm also using checkpointing.
> If I deploy a new pipeline with the same id's on the nodes, will the
> checkpoint be atomatically loaded and the state restored?
>
> -Thanks
> Simon
>

Mime
View raw message