kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Oleg Kuznetsov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-5696) SourceConnector does not commit offset on rebalance
Date Thu, 24 Aug 2017 02:41:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5696?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16139481#comment-16139481

Oleg Kuznetsov commented on KAFKA-5696:

[~Yohan123] Main issue was not resolved - we have now understanding that workflow of one connector
affects the other, and other connectors might be writing data at the moment of restart.

Not sure how it can be resolved at some levels, could you clarify?

> SourceConnector does not commit offset on rebalance
> ---------------------------------------------------
>                 Key: KAFKA-5696
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5696
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>            Reporter: Oleg Kuznetsov
>            Priority: Critical
>              Labels: newbie
> I'm running SourceConnector, that reads files from storage and put data in kafka. I want,
in case of reconfiguration, offsets to be flushed. 
> Say, a file is completely processed, but source records are not yet committed and in
case of reconfiguration their offsets might be missing in store.
> Is it possible to force committing offsets on reconfiguration?

This message was sent by Atlassian JIRA

View raw message