Mailing-List: contact jira-help@kafka.apache.org; run by ezmlm
Precedence: bulk
Reply-To: jira@kafka.apache.org
Date: Mon, 6 Nov 2017 04:10:00 +0000 (UTC)
From: "Eugene Burd (JIRA)" <jira@apache.org>
To: jira@kafka.apache.org
Message-ID: <JIRA.13113156.1509400899000.155411.1509941400183@Atlassian.JIRA>
In-Reply-To: <JIRA.13113156.1509400899000@Atlassian.JIRA>
References: <JIRA.13113156.1509400899000@Atlassian.JIRA> <JIRA.13113156.1509400899800@jira-lw-us.apache.org>
Subject: [jira] [Commented] (KAFKA-6148) ClassCastException in BigQuery
 connector
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit
archived-at: Mon, 06 Nov 2017 04:10:05 -0000


    [ https://issues.apache.org/jira/browse/KAFKA-6148?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16239880#comment-16239880 ] 

Eugene Burd commented on KAFKA-6148:
------------------------------------

[~ewencp] You are correct, there is not git check in with 0.11.  After encountering this error, I figured it was a version difference between the plugin path & the classpath and upgraded the bigquery connector to .11 locally.  This still didn't remedy the problem.  

Additionally, you are correct - the code is unnecessary and commenting it out resolved the issue.  I will work on a PR for that project.  

[~kkonstantine] - that makes sense.  I will try modifying the gradle script to exclude the connect jars.  Regarding your approach to the fix, I prefer a harder failure that is verbose with what to do over a failure later on (on offset commit) that is not verbose.  

Thanks both for your help. 

> ClassCastException in BigQuery connector
> ----------------------------------------
>
>                 Key: KAFKA-6148
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6148
>             Project: Kafka
>          Issue Type: Bug
>          Components: KafkaConnect
>            Reporter: Eugene Burd
>            Assignee: Konstantine Karantasis
>
> I am trying to run a com.wepay.kafka.connect.bigquery.BigQuerySinkConnector connector, but getting the following exception.  
> [2017-10-30 21:48:49,007] ERROR WorkerSinkTask{id=bigquery-connector-log-0} Offset commit failed, rewinding to last committed offsets (org.apache.kafka.connect.runtime.WorkerSinkTask:311)
> java.lang.ClassCastException: org.apache.kafka.clients.consumer.OffsetAndMetadata cannot be cast to org.apache.kafka.clients.consumer.OffsetAndMetadata
> 	at com.wepay.kafka.connect.bigquery.BigQuerySinkTask.updateOffsets(BigQuerySinkTask.java:107)
> 	at com.wepay.kafka.connect.bigquery.BigQuerySinkTask.flush(BigQuerySinkTask.java:96)
> 	at org.apache.kafka.connect.sink.SinkTask.preCommit(SinkTask.java:117)
> 	at org.apache.kafka.connect.runtime.WorkerSinkTask.commitOffsets(WorkerSinkTask.java:305)
> 	at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:164)
> 	at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:148)
> 	at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:146)
> 	at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:190)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:748)
> [2017-10-30 21:48:49,012] ERROR Commit of WorkerSinkTask{id=bigquery-connector-log-0} offsets threw an unexpected exception:  (org.apache.kafka.connect.runtime.WorkerSinkTask:205)
> java.lang.ClassCastException: org.apache.kafka.clients.consumer.OffsetAndMetadata cannot be cast to org.apache.kafka.clients.consumer.OffsetAndMetadata
> 	at com.wepay.kafka.connect.bigquery.BigQuerySinkTask.updateOffsets(BigQuerySinkTask.java:107)
> 	at com.wepay.kafka.connect.bigquery.BigQuerySinkTask.flush(BigQuerySinkTask.java:96)
> 	at org.apache.kafka.connect.sink.SinkTask.preCommit(SinkTask.java:117)
> 	at org.apache.kafka.connect.runtime.WorkerSinkTask.commitOffsets(WorkerSinkTask.java:305)
> 	at org.apache.kafka.connect.runtime.WorkerSinkTask.iteration(WorkerSinkTask.java:164)
> 	at org.apache.kafka.connect.runtime.WorkerSinkTask.execute(WorkerSinkTask.java:148)
> 	at org.apache.kafka.connect.runtime.WorkerTask.doRun(WorkerTask.java:146)
> 	at org.apache.kafka.connect.runtime.WorkerTask.run(WorkerTask.java:190)
> 	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> 	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> 	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> 	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> 	at java.lang.Thread.run(Thread.java:748)
> I have checked the version number of kafka client in the plug in and kafka connect itself and they are the same.  
> - kafka-clients-0.11.0.0.jar matches
> I am still suspecting a type of versioning issue.  Do you have any advice? 
> Thanks. 


--
This message was sent by Atlassian JIRA
(v6.4.14#64029)