flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai" <tzuli...@apache.org>
Subject Re: Kinesis connector - Jackson issue
Date Tue, 26 Sep 2017 14:43:01 GMT
Hi Tomasz,

Yes, dependency clashes may surface when executing actual job runs on clusters.

A few things to probably check first:
- Have you built Flink or the Kinesis connector with Maven version 3.3 or above? If yes, try
using a lower version, as 3.3+ results in some shading issues when used to build Flink.
- I’m not sure if the Kinesis client has a Jackson dependency, but you could also try checking
if your application pulls in a conflicting Jackson version (with Flink, which uses 2.7.4)
via some other dependency.

Cheers,
Gordon

On 26 September 2017 at 4:28:27 PM, Tomasz Dobrzycki (dobrzycki.tomasz@gmail.com) wrote:

Hi guys,  

I'm working with Kinesis connector and currently trying to solve a  
bizarre issue.  
I had problems with Kinesis and httpcomponents which I was able to  
solve using steps shown in:  
https://github.com/apache/flink/pull/4150/commits/9b539470ac308d7af9df9a70792aa1fa8c6995fc
 

That did the trick and I am able to run my code successfully via  
IntelliJ. I am connecting to Kinesis stream hosted on AWS and reading  
messages just fine.  

Unfortunately that is not true for running Flink via command line  
scripts. I get this error when running start-local.sh into flink run  
... :  
Caused by: java.lang.ClassNotFoundException:  
com.fasterxml.jackson.dataformat.cbor.CBORFactory  

I have built my Kinesis connector and installed it via mvn install. Am  
I missing some steps? I'm assuming that my code is fine given that I'm  
able to run it through IntelliJ.  

Anyone faced this problem or maybe some solution comes to your mind?  

Cheers  
Tomasz  

Mime
View raw message