kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Richard Yu (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (KAFKA-5696) SourceConnector does not commit offset on rebalance
Date Tue, 05 Sep 2017 02:25:02 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16153047#comment-16153047
] 

Richard Yu edited comment on KAFKA-5696 at 9/5/17 2:24 AM:
-----------------------------------------------------------

There is the reconfigure method that was originally defined in the Connector abstract class.
Knowing that SourceConnector,also an abstract class,extends Connector, the method can be overridden,
so that in the 
process of reconfiguration, the offsets could be committed.


was (Author: yohan123):
There is the reconfigure method that was originally defined in the Connector abstract class.
Knowing that SourceConnector--also an abstract class--extends Connector, the method can be
overridden, so that in the 
process of reconfiguration, the offsets could be committed.

> SourceConnector does not commit offset on rebalance
> ---------------------------------------------------
>
>                 Key: KAFKA-5696
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5696
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>            Reporter: Oleg Kuznetsov
>            Priority: Critical
>              Labels: newbie
>
> I'm running SourceConnector, that reads files from storage and put data in kafka. I want,
in case of reconfiguration, offsets to be flushed. 
> Say, a file is completely processed, but source records are not yet committed and in
case of reconfiguration their offsets might be missing in store.
> Is it possible to force committing offsets on reconfiguration?



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message