kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Randall Hauch (JIRA)" <j...@apache.org>
Subject [jira] [Resolved] (KAFKA-6661) Sink connectors that explicitly 'resume' topic partitions can resume a paused task
Date Mon, 19 Mar 2018 23:19:00 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-6661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Randall Hauch resolved KAFKA-6661.
    Resolution: Fixed

> Sink connectors that explicitly 'resume' topic partitions can resume a paused task
> ----------------------------------------------------------------------------------
>                 Key: KAFKA-6661
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6661
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>    Affects Versions:,,, 1.0.0
>            Reporter: Randall Hauch
>            Assignee: Randall Hauch
>            Priority: Critical
>             Fix For:,,,, 1.0.2, 1.1.1
> Sink connectors are allowed to use the {{SinkTaskContext}}'s methods to explicitly pause
and resume topic partitions. This is useful when connectors need additional time processing
the records for specific topic partitions (e.g., the external system has an outage).
> However, when the sink connector has been paused via the REST API, the worker for the
sink tasks pause the consumer. When the connector is polled, the poll request might timeout
and return no records. Connect then calls the task's {{put(...)}} method (with no records),
and this allows the task to optionally call any of the {{SinkTaskContext}}'s pause or resume
methods. If it calls resume, this will unexpectedly resume the paused consumer, causing the
consumer to return messages and the connector to process those messages --  despite the connector
still being paused.
> This is reported against 1.0, but the affected code has not been changed since at least
> A workaround is to remove rather than pause a connector. It's inconvenient, but it works.

This message was sent by Atlassian JIRA

View raw message