kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ewen Cheslack-Postava (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-2480) Handle non-CopycatExceptions from
Date Thu, 27 Aug 2015 17:10:45 GMT
Ewen Cheslack-Postava created KAFKA-2480:

             Summary: Handle non-CopycatExceptions from
                 Key: KAFKA-2480
                 URL: https://issues.apache.org/jira/browse/KAFKA-2480
             Project: Kafka
          Issue Type: Sub-task
          Components: copycat
            Reporter: Ewen Cheslack-Postava
            Assignee: Ewen Cheslack-Postava

Currently we catch Throwable in WorkerSinkTask, but we just log the exception. This can lead
to data loss because it indicates the messages in the {{put(records)}} call probably were
not handled properly. We need to decide what the policy for handling these types of exceptions
should be -- try repeating the same records again, risking duplication? or skip them, risking
loss? or kill the task immediately and require intervention since it's unclear what happened?

SourceTasks don't have the same concern -- they can throw other exceptions and as long as
we catch them, it is up to the connector to ensure that it does not lose data as a result.

This message was sent by Atlassian JIRA

View raw message