flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Oscar Westra van Holthe - Kind <os...@westravanholthe.nl>
Subject Re: About the issue caused by flink's dependency jar package submission method
Date Wed, 21 Nov 2018 10:17:22 GMT
On Wed, 21 Nov 2018 at 05:26, clay4444 <clay4megtr@gmail.com> wrote:

> hi
>
> I have checked all the dependences, and don't find the jar with different
> version, so ,I double the way to submit jar has some issue? my commend is
> like this:
>

Did you also check the runtime dependencies where the code is run? Because
the only way that two classes with the same name are not the same is if
they are loaded with two different classloaders.

I suspect your runtime loads the LinkedMap via a different classloader than
FlinkKafkaConsumerBase, and that other classloader (probably a parent) also
has the LinkedMap. Hence the problem.

[...]

Issues like these have a similar origin, but now you have two versions of
kafka on your classpath:

> Caused by: org.apache.kafka.common.KafkaException:

org.apache.kafka.common.serialization.ByteArrayDeserializer is not an
> instance of org.apache.kafka.common.serialization.Deserializer
>         at
>
> org.apache.kafka.common.config.AbstractConfig.getConfiguredInstance(AbstractConfig.java:248)
>         at
>
> org.apache.kafka.clients.consumer.KafkaConsumer.<init>(KafkaConsumer.java:673)
>         ... 11 more
>

[...]


>  but in a fat jar, these work all good
>

That's odd.. that means that in that case, your classpath is different. If
you can find out how, solving the exceptions above becomes easy.

Kind regards,
Oscar


-- 
Oscar Westra van Holthe - Kind
Website: http://oscar.westravanholthe.nl/ | Email: oscar@westravanholthe.nl

Mime
View raw message