kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Richard Yu (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-5696) SourceConnector does not commit offset on rebalance
Date Tue, 05 Sep 2017 02:25:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16153047#comment-16153047

Richard Yu commented on KAFKA-5696:

There is the reconfigure method that was originally defined in the Connector abstract class.
Knowing that SourceConnector--also an abstract class--extends Connector, the method can be
overridden, so that in the 
process of reconfiguration, the offsets could be committed.

> SourceConnector does not commit offset on rebalance
> ---------------------------------------------------
>                 Key: KAFKA-5696
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5696
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>            Reporter: Oleg Kuznetsov
>            Priority: Critical
>              Labels: newbie
> I'm running SourceConnector, that reads files from storage and put data in kafka. I want,
in case of reconfiguration, offsets to be flushed. 
> Say, a file is completely processed, but source records are not yet committed and in
case of reconfiguration their offsets might be missing in store.
> Is it possible to force committing offsets on reconfiguration?

This message was sent by Atlassian JIRA

View raw message