kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Behrang Saeedzadeh (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-5827) Allow configuring Kafka sink connectors to start processing records from the end of topics
Date Sun, 03 Sep 2017 14:11:00 GMT
Behrang Saeedzadeh created KAFKA-5827:
-----------------------------------------

             Summary: Allow configuring Kafka sink connectors to start processing records
from the end of topics
                 Key: KAFKA-5827
                 URL: https://issues.apache.org/jira/browse/KAFKA-5827
             Project: Kafka
          Issue Type: Improvement
          Components: KafkaConnect
            Reporter: Behrang Saeedzadeh


As far as I can see, Kafka connectors start exporting data of a topic from its beginning.
We have a topic that contains a few million old records that we don't need but we would like
to start exporting new records that are added to the topic.

Basically:

* When the connector is started for the first time and it does not have a current offset stored,
it should start consuming data from the end of topics
* When the connector is restarted and has a current offset stored somewhere, it should start
from that offset




--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message