flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: Error with Avro/Kyro after upgrade to Flink 1.4
Date Wed, 24 Jan 2018 09:47:38 GMT
I would assume that once you build Flink with correct shading and add all
the required Jackson dependencies to your application jar, all works fine.

On Jan 24, 2018 10:23, "Stephan Ewen" <sewen@apache.org> wrote:

> Hi!
>
> I have a patch coming up for Verify Error
>
> Concerning the Jackson Error - did you build Flink yourself? It looks like
> flink-dist was not built properly, the shading is incorrect.
> The class "com.fasterxml.jackson.databind.ObjectMapper" should not be in
> the jar in an unshaded manner.
>
> My first guess is you build Flink yourself with Maven version >= 3.3.0
>
> Please see here: https://ci.apache.org/projects/flink/flink-docs-
> release-1.4/start/building.html#dependency-shading
>
> Stephan
>
>
> On Tue, Jan 23, 2018 at 5:21 PM, Edward <egbuck@hotmail.com> wrote:
>
>> Thanks for the follow-up Stephan.
>>
>> I have been running this job from a built jar file which was submitted to
>> an
>> existing Flink 1.4 cluster, not from within the IDE. Interestingly, I am
>> now
>> getting the same error when any of the following 3 conditions are true:
>> 1. I run the job on a local cluster from within my IDE
>> 2. I run the job on a cluster where "classloader.resolve-order:
>> parent-first"
>> 3. I build the uber jar file without including flink-java,
>> flink-streaming-java and flink-clients (I changed those to "provided" as
>> you
>> suggested, so they aren't in my jar)
>>
>> If any of those 3 cases are true, I get a new NoClassDefFoundError. This
>> error is caused because com.fasterxml.jackson.databind.ObjectMapper is
>> present in flink-dist_2.11-1.4.0.jar, but
>> com.fasterxml.jackson.databind.SerializationConfig is not (only the
>> shaded
>> version:
>> org.apache.flink.shaded.jackson2.com.fasterxml.jackson.
>> databind.SerializationConfig)
>>
>> java.lang.NoClassDefFoundError: Could not initialize class
>> com.fasterxml.jackson.databind.SerializationConfig
>>         at
>> com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:558)
>>         at
>> com.fasterxml.jackson.databind.ObjectMapper.<init>(ObjectMapper.java:474)
>>         at
>> com.mycom.datapipeline.common.client.UmsCientFactory.getUser
>> MappingServiceClient(UmsCientFactory.java:31)
>>         at
>> com.mycom.datapipeline.flink.udf.UserLookupFunctionBase.open
>> (UserLookupFunctionBase.java:78)
>>         at
>> org.apache.flink.api.common.functions.util.FunctionUtils.ope
>> nFunction(FunctionUtils.java:36)
>>         at
>> org.apache.flink.streaming.api.operators.AbstractUdfStreamOp
>> erator.open(AbstractUdfStreamOperator.java:102)
>>         at
>> org.apache.flink.streaming.runtime.tasks.StreamTask.openAllO
>> perators(StreamTask.java:393)
>>         at
>> org.apache.flink.streaming.runtime.tasks.StreamTask.invoke(
>> StreamTask.java:254)
>>         at org.apache.flink.runtime.taskmanager.Task.run(Task.java:718)
>>         at java.lang.Thread.run(Thread.java:748)
>>
>> I understand why this is happening in the case of the parent-first
>> classloader, but I can't understand why it's happening when I exclude
>> flink-java from my job's uber jar file -- in that 2nd case, I would expect
>> the job's child classloader to be used, which contains both of those
>> fasterxml classes.
>>
>>
>>
>> --
>> Sent from: http://apache-flink-user-mailing-list-archive.2336050.n4.
>> nabble.com/
>>
>
>

Mime
View raw message