kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stelios BOURAZANIS (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4107) Support offset reset capability in Kafka Connect
Date Fri, 22 Sep 2017 07:11:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16176026#comment-16176026

Stelios BOURAZANIS commented on KAFKA-4107:

Yes it satisfy my requirements.
 Right now in order to satisfy the reset requirement:
 1. I Pause the connector
 2. I use a producer to produce the "reset" message on the connect-offset topic using a table
key that sends it to the right partition.
 3. Delete the connector
 4. Create the connector again.

This is not optimal as all tasks reading from other tables also stop.
But I cannot not do better at the moment.

Only on create connector reads the external produced message.
*So it is crucial that the new tool makes the offset reset on the fly.*

> Support offset reset capability in Kafka Connect
> ------------------------------------------------
>                 Key: KAFKA-4107
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4107
>             Project: Kafka
>          Issue Type: Improvement
>          Components: KafkaConnect
>            Reporter: Jason Gustafson
> It would be useful in some cases to be able to reset connector offsets. For example,
if a topic in Kafka corresponding to a source database is accidentally deleted (or deleted
because of corrupt data), an administrator may want to reset offsets and reproduce the log
from the beginning. It may also be useful to have support for overriding offsets, but that
seems like a less likely use case.

This message was sent by Atlassian JIRA

View raw message