flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: Reading from latest offset in kafka consumer on restart
Date Wed, 03 Aug 2016 09:42:59 GMT
Checkpointing starts the consumer where it left off in case the job fails
and recovers.
If you explicitly cancel a job and start a new job (same jar), the new job
will not start from a checkpoint, but from blank state.


On Wed, Aug 3, 2016 at 11:37 AM, Janardhan Reddy <
janardhan.reddy@olacabs.com> wrote:

> I mean in case of chekpointing, won't the consumer start from where it
> previously left ?
>
> On Wed, Aug 3, 2016 at 3:06 PM, Janardhan Reddy <
> janardhan.reddy@olacabs.com> wrote:
>
>> How would checkpointing affect the offset.
>>
>> On Wed, Aug 3, 2016 at 3:03 PM, Stephan Ewen <sewen@apache.org> wrote:
>>
>>> When you cancel and restart a Flink job (without a savepoint), it does
>>> not use the checkpoint data, and uses the behavior you defined in the Kafka
>>> consumer to decide where to start from (consumer group, latest, or
>>> earliest).
>>>
>>> On Wed, Aug 3, 2016 at 11:26 AM, Janardhan Reddy <
>>> janardhan.reddy@olacabs.com> wrote:
>>>
>>>> Hi,
>>>>
>>>> Is there a way to read from latest offset in kafka consumer on restart.
>>>> Or can we somehow start flink ignoring previous checkpointed data.
>>>>
>>>> Thanks
>>>>
>>>
>>>
>>
>

Mime
View raw message