kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Randall Hauch (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-3821) Allow Kafka Connect source tasks to produce offset without writing to topics
Date Thu, 01 Feb 2018 14:52:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-3821?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16348676#comment-16348676
] 

Randall Hauch commented on KAFKA-3821:
--------------------------------------

[~gunnar.morling], that'd be great. The [CONTRIBUTING.md|https://github.com/apache/kafka/blob/trunk/CONTRIBUTING.md]
file in the AK git repo has the links to the documentation about how/when to create a KIP
and what the process is. Let me know if that's not enough.

> Allow Kafka Connect source tasks to produce offset without writing to topics
> ----------------------------------------------------------------------------
>
>                 Key: KAFKA-3821
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3821
>             Project: Kafka
>          Issue Type: Improvement
>          Components: KafkaConnect
>    Affects Versions: 0.9.0.1
>            Reporter: Randall Hauch
>            Priority: Major
>              Labels: needs-kip
>             Fix For: 1.2.0
>
>
> Provide a way for a {{SourceTask}} implementation to record a new offset for a given
partition without necessarily writing a source record to a topic.
> Consider a connector task that uses the same offset when producing an unknown number
of {{SourceRecord}} objects (e.g., it is taking a snapshot of a database). Once the task completes
those records, the connector wants to update the offsets (e.g., the snapshot is complete)
but has no more records to be written to a topic. With this change, the task could simply
supply an updated offset.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message