kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "William R. Speirs (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-6490) JSON SerializationException Stops Connect
Date Fri, 26 Jan 2018 15:40:00 GMT
William R. Speirs created KAFKA-6490:

             Summary: JSON SerializationException Stops Connect
                 Key: KAFKA-6490
                 URL: https://issues.apache.org/jira/browse/KAFKA-6490
             Project: Kafka
          Issue Type: Bug
          Components: KafkaConnect
    Affects Versions: 1.0.0
            Reporter: William R. Speirs

If you configure KafkaConnect to parse JSON messages, and you send it a non-JSON message,
the SerializationException message will bubble up to the top, and stop KafkaConnect. While
I understand sending non-JSON to a JSON serializer is a bad idea, I think that a single malformed
message stopping all of KafkaConnect is even worse.

The data exception is thrown here: [https://github.com/apache/kafka/blob/trunk/connect/json/src/main/java/org/apache/kafka/connect/json/JsonConverter.java#L305]
>From the call here: [https://github.com/apache/kafka/blob/trunk/connect/runtime/src/main/java/org/apache/kafka/connect/runtime/WorkerSinkTask.java#L476]

This bubbles all the way up to the top, and KafkaConnect simply stops with the message: {{ERROR
WorkerSinkTask\{id=elasticsearch-sink-0} Task threw an uncaught and unrecoverable exception

Thoughts on adding a {{try/catch}} around the {{for}} loop in WorkerSinkTask's {{convertMessages}}
so messages that don't properly parse are logged, but simply ignored? This way KafkaConnect
can keep working even when it encounters a message it cannot decode?

This message was sent by Atlassian JIRA

View raw message