kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Bartłomiej Tartanus (JIRA) <j...@apache.org>
Subject [jira] [Commented] (KAFKA-5996) JsonConverter generates "Mismatching schema" DataException
Date Sat, 03 Feb 2018 22:57:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16351567#comment-16351567
] 

Bartłomiej Tartanus commented on KAFKA-5996:
--------------------------------------------

I think this issue might be related to mine. I am also getting this error despite the fact
that schemas are equal. Looks like this is bug in this line:

[https://github.com/apache/kafka/blob/4c48942f9d9e1428e21f934746cb7ce22b3df746/connect/json/src/main/java/org/apache/kafka/connect/json/JsonConverter.java#L676]

schema and struct schema are compared using != and not equals method which causes this exception
to be thrown incorrectly when schemas are indeed equal. 

> JsonConverter generates "Mismatching schema" DataException
> ----------------------------------------------------------
>
>                 Key: KAFKA-5996
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5996
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>    Affects Versions: 0.10.0.0, 0.10.0.1, 0.10.1.0, 0.10.1.1, 0.10.2.0, 0.10.2.1, 0.11.0.0
>            Reporter: Yuqi Li
>            Priority: Major
>
> Run connect-standalone.sh with original connect-standalone.properties and custom jdbc
source connect configuration.
> part of the jdbc source connect configuration:
> connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
> query=Select * from ......
> poll.interval.ms=45000
> transforms=MakeMap, InsertSourceId, InsertSourceEvent
> transforms.MakeMap.type=org.apache.kafka.connect.transforms.HoistField$Value
> transforms.MakeMap.field=emailRequest
> transforms.InsertSourceId.type=org.apache.kafka.connect.transforms.InsertField$Value
> transforms.InsertSourceId.static.field=id
> transforms.InsertSourceId.static.value=1
> transforms.InsertSourceEvent.type=org.apache.kafka.connect.transforms.InsertField$Value
> transforms.InsertSourceEvent.static.field=event
> transforms.InsertSourceEvent.static.value=REMINDER_EMAIL
> When the source connect starts, the first iteration(executing the query) runs ok, but
45 seconds later, it throws following exception:
> org.apache.kafka.connect.errors.DataException: Mismatching schema.
>         at org.apache.kafka.connect.json.JsonConverter.convertToJson(JsonConverter.java:647)
>         at org.apache.kafka.connect.json.JsonConverter.convertToJson(JsonConverter.java:650)
>         at org.apache.kafka.connect.json.JsonConverter.convertToJsonWithEnvelope(JsonConverter.java:537)
>         at org.apache.kafka.connect.json.JsonConverter.fromConnectData(JsonConverter.java:290)
>         at org.apache.kafka.connect.runtime.WorkerSourceTask.sendRecords(WorkerSourceTask.java:198)
>         at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:168)
>         at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:146)
>         at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:190)
>         at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
>         at java.util.concurrent.FutureTask.run(Unknown Source)
>         at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
>         at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
>         at java.lang.Thread.run(Unknown Source)
> [2017-09-29 13:59:18,361] ERROR Task is being killed and will not recover until manually
restarted (org.apache.kafka.connect.runtime.WorkerTask:149)
> The SoureTask is being killed due to the exception, and the source connector is dead.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message