flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Robert Metzger <rmetz...@apache.org>
Subject Re: Flink 0.10.2 and Kafka 0.8.1
Date Mon, 18 Apr 2016 10:36:20 GMT
Hi,
did you check your user jar if it contains the Kafka classes?
Are you building a fat jar? Are you manually excluding any dependencies?

Flink's 0.10.2 Kafka connector depends on Kafka 0.8.2.0 [1] which in turn
depends on kafka-clients 0.8.2.0 [2]. And the "kafka-clients" dependency
also contains the org.apache.kafka.common.Node class (the LegacyFetcher
needs that class).


[1]
https://repo1.maven.org/maven2/org/apache/flink/flink-connector-kafka/0.10.2/flink-connector-kafka-0.10.2.pom
[2]
https://repo1.maven.org/maven2/org/apache/kafka/kafka_2.10/0.8.2.0/kafka_2.10-0.8.2.0.pom

On Mon, Apr 18, 2016 at 7:42 AM, Balaji Rajagopalan <
balaji.rajagopalan@olacabs.com> wrote:

> I had fought with 0.8.0.2 kafka and flink 0.10.2 scala version 2.11, was
> never able to get it working confounded with noclassdeffounderror, moved to
> flink 1.0.0 with kafka 0.8.0.2  scala version 2.11 things worked for me, if
> moving to flink 1.0.0 is an option for you do so.
>
> balaji
>
> On Mon, Apr 18, 2016 at 3:19 AM, Robert Schmidtke <ro.schmidtke@gmail.com>
> wrote:
>
>> Hi everyone,
>>
>> I have a Kafka cluster running on version 0.8.1, hence I'm using the
>> FlinkKafkaConsumer081. When running my program, I saw a
>> NoClassDefFoundError for org.apache.kafka.common.Node. So I packaged my
>> binaries according to
>> https://ci.apache.org/projects/flink/flink-docs-release-0.10/apis/cluster_execution.html#linking-with-modules-not-contained-in-the-binary-distribution,
>> however I'm still seeing the error.
>>
>> I played around a bit and it turns out I have to package kafka-clients v.
>> 0.8.2.0 instead of kafka_2.10 v. 0.8.1 with my program. Is there an error
>> in the documentation or have I not figured out something properly?
>>
>> Thanks!
>> Robert
>>
>> --
>> My GPG Key ID: 336E2680
>>
>
>

Mime
View raw message