kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eugene Burd (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-6148) ClassCastException in BigQuery connector
Date Mon, 06 Nov 2017 04:10:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16239880#comment-16239880
] 

Eugene Burd commented on KAFKA-6148:
------------------------------------

[~ewencp] You are correct, there is not git check in with 0.11.  After encountering this error,
I figured it was a version difference between the plugin path & the classpath and upgraded
the bigquery connector to .11 locally.  This still didn't remedy the problem.  

Additionally, you are correct - the code is unnecessary and commenting it out resolved the
issue.  I will work on a PR for that project.  

[~kkonstantine] - that makes sense.  I will try modifying the gradle script to exclude the
connect jars.  Regarding your approach to the fix, I prefer a harder failure that is verbose
with what to do over a failure later on (on offset commit) that is not verbose.  

Thanks both for your help. 

> ClassCastException in BigQuery connector
> ----------------------------------------
>
>                 Key: KAFKA-6148
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6148
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>            Reporter: Eugene Burd
>            Assignee: Konstantine Karantasis
>
> I am trying to run a com.wepay.kafka.connect.bigquery.BigQuerySinkConnector connector,
but getting the following exception.  
> [2017-10-30 21:48:49,007] ERROR WorkerSinkTask{id=bigquery-connector-log-0} Offset commit
failed, rewinding to last committed offsets (org.apache.kafka.connect.runtime.WorkerSinkTask:311)
> java.lang.ClassCastException: org.apache.kafka.clients.consumer.OffsetAndMetadata cannot
be cast to org.apache.kafka.clients.consumer.OffsetAndMetadata
> 	at com.wepay.kafka.connect.bigquery.BigQuerySinkTask.updateOffsets(BigQuerySinkTask.java:107)
> 	at com.wepay.kafka.connect.bigquery.BigQuerySinkTask.flush(BigQuerySinkTask.java:96)
> 	at org.apache.kafka.connect.sink.SinkTask.preCommit(SinkTask.java:117)
> 	at org.apache.kafka.connect.runtime.WorkerSinkTask.commitOffsets(WorkerSinkTask.java:305)
> 	at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:164)
> 	at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:148)
> 	at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:146)
> 	at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:190)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:748)
> [2017-10-30 21:48:49,012] ERROR Commit of WorkerSinkTask{id=bigquery-connector-log-0}
offsets threw an unexpected exception:  (org.apache.kafka.connect.runtime.WorkerSinkTask:205)
> java.lang.ClassCastException: org.apache.kafka.clients.consumer.OffsetAndMetadata cannot
be cast to org.apache.kafka.clients.consumer.OffsetAndMetadata
> 	at com.wepay.kafka.connect.bigquery.BigQuerySinkTask.updateOffsets(BigQuerySinkTask.java:107)
> 	at com.wepay.kafka.connect.bigquery.BigQuerySinkTask.flush(BigQuerySinkTask.java:96)
> 	at org.apache.kafka.connect.sink.SinkTask.preCommit(SinkTask.java:117)
> 	at org.apache.kafka.connect.runtime.WorkerSinkTask.commitOffsets(WorkerSinkTask.java:305)
> 	at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:164)
> 	at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:148)
> 	at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:146)
> 	at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:190)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:748)
> I have checked the version number of kafka client in the plug in and kafka connect itself
and they are the same.  
> - kafka-clients-0.11.0.0.jar matches
> I am still suspecting a type of versioning issue.  Do you have any advice? 
> Thanks. 



--
This message was sent by Atlassian JIRA
(v6.4.14#64029)

Mime
View raw message