kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "ASF GitHub Bot (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-5891) Cast transformation fails if record schema contains timestamp field
Date Thu, 01 Mar 2018 20:56:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5891?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16382629#comment-16382629

ASF GitHub Bot commented on KAFKA-5891:

maver1ck opened a new pull request #4633: [KAFKA-5891] Proper handling of LogicalTypes in
URL: https://github.com/apache/kafka/pull/4633
   Currently logical types are dropped during Cast Transformation.
   This patch fixes this behaviour.
   ### Committer Checklist (excluded from commit message)
   - [X] Verify design and implementation 
   - [X] Verify test coverage and CI build status
   - [X] Verify documentation (including upgrade notes)

This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:

> Cast transformation fails if record schema contains timestamp field
> -------------------------------------------------------------------
>                 Key: KAFKA-5891
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5891
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>    Affects Versions:
>            Reporter: Artem Plotnikov
>            Priority: Major
> I have the following simple type cast transformation:
> {code}
> name=postgresql-source-simple
> connector.class=io.confluent.connect.jdbc.JdbcSourceConnector
> tasks.max=1
> connection.url=jdbc:postgresql://localhost:5432/testdb?user=postgres&password=mysecretpassword
> query=SELECT 1::INT as a, '2017-09-14 10:23:54'::TIMESTAMP as b
> transforms=Cast
> transforms.Cast.type=org.apache.kafka.connect.transforms.Cast$Value
> transforms.Cast.spec=a:boolean
> mode=bulk
> topic.prefix=clients
> {code}
> Which fails with the following exception in runtime:
> {code}
> [2017-09-14 16:51:01,885] ERROR Task postgresql-source-simple-0 threw an uncaught and
unrecoverable exception (org.apache.kafka.connect.runtime.WorkerTask:148)
> org.apache.kafka.connect.errors.DataException: Invalid Java object for schema type INT64:
class java.sql.Timestamp for field: "null"
> 	at org.apache.kafka.connect.data.ConnectSchema.validateValue(ConnectSchema.java:239)
> 	at org.apache.kafka.connect.data.ConnectSchema.validateValue(ConnectSchema.java:209)
> 	at org.apache.kafka.connect.data.Struct.put(Struct.java:214)
> 	at org.apache.kafka.connect.transforms.Cast.applyWithSchema(Cast.java:152)
> 	at org.apache.kafka.connect.transforms.Cast.apply(Cast.java:108)
> 	at org.apache.kafka.connect.runtime.TransformationChain.apply(TransformationChain.java:38)
> 	at org.apache.kafka.connect.runtime.WorkerSourceTask.sendRecords(WorkerSourceTask.java:190)
> 	at org.apache.kafka.connect.runtime.WorkerSourceTask.execute(WorkerSourceTask.java:168)
> 	at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:146)
> 	at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:190)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
>   at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:745)
> {code}
> If I remove the  transforms.* part of the connector it will work correctly. Actually,
it doesn't really matter which types I use in the transformation for field 'a', just the existence
of a timestamp field brings the exception.

This message was sent by Atlassian JIRA

View raw message