flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Dominik Safaric <dominiksafa...@gmail.com>
Subject Flink Kafka connector
Date Thu, 03 Nov 2016 07:05:47 GMT
Dear all,

Although the Flink 1.2 version will rollout a Flink Kafka 0.10.x connector, I want to use
the Flink 0.9 connector in conjunction with the 0.10.x versions. 

The reason behind this is because we are currently evaluating Flink part of an empirical research,
hence a stable release is required. In addition, the reason why we have the requirement of
using the Kafka 0.10.x versions is because since the 0.10.0 Kafka supports consumer and producer
interceptors and message timestamps.

To make the 0.9 connector support Kafka version e.g. 0.10.0 for example, so far I’ve changed
the Flink Kafka 0.9 connector dependency to the required Kafka version and build the project.
However, as I imported the jar and added the source to the StreamExecutionEnvironment a type
error occurred stating that the addSource function requires a class deriving from the SourceFunction

Hence, what have gone wrong during the build? I assume a dependency issue.

Next, I’ve tried just simply overriding the dependencies of the Flink Kafka connector within
the project pom.xml, however there is obviously a slight API mismatch hence this cannot be

I would really appreciate if anyone could provide some guidance once how to successfully build
the Flink Kafka connector supporting Kafka 0.10.x versions. 

Thanks in advance,
View raw message