flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: Flink Kafka from Time Offset
Date Mon, 30 May 2016 12:28:09 GMT
Hi Simon,

Timestamp-awareness has been added to Kafka 0.10 only [1]. I'm not sure if
the 0.9 connector code of Flink will support Kafka 0.10 immediately.
Another way of handling the issue would be
- Implementing a custom offset-metadata system (storing the timestamp for
some of the offsets (say every minute or so))
- Implement this feature into Flink's Kafka connector:

Regarding your second question: What "id"s do you mean?
If you are not using savepoints, the checkpoint state will not be
automatically restored.


[1] https://issues.apache.org/jira/browse/KAFKA-2511

On Mon, May 30, 2016 at 1:45 PM, simon peyer <simon.peyer@soom-it.ch> wrote:

> Hi together
> I'm using flink 1.0.1 and a FlinkKafkaConsumer09.
> I'm very interested in getting data from a specific Time offset in Kafka.
> Is there a property which can do this?
> Or is there another way of handling such issues?
> I'm also using checkpointing.
> If I deploy a new pipeline with the same id's on the nodes, will the
> checkpoint be atomatically loaded and the state restored?
> -Thanks
> Simon

View raw message