flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Stephan Ewen <se...@apache.org>
Subject Re: AWS exception serialization problem
Date Wed, 08 Mar 2017 20:41:22 GMT
@Bruno: How are you running Flink? On yarn, standalone, mesos, docker?

On Wed, Mar 8, 2017 at 2:13 PM, Bruno Aranda <brunoaranda@gmail.com> wrote:

> Hi,
>
> We have seen something similar in Flink 1.2. We have an operation that
> parses some JSON, and when it fails to parse it, we can see the
> ClassNotFoundException for the relevant exception (in our case
> JsResultException from the play-json library). The library is indeed in the
> shaded JAR, otherwise we would not be able to parse the JSON.
>
> Cheers,
>
> Bruno
>
> On Wed, 8 Mar 2017 at 12:57 Tzu-Li (Gordon) Tai <tzulitai@apache.org>
> wrote:
>
>> Hi Shannon,
>>
>> Just to clarify:
>>
>> From the error trace, it seems like that the messages fetched from Kafka
>> are serialized `AmazonS3Exception`s, and you’re emitting a stream of
>> `AmazonS3Exception` as records from FlinkKafkaConsumer?
>> Is this correct? If so, I think we should just make sure that the
>> `com.amazonaws.services.s3.model.AmazonS3Exception` class exists in the
>> user fat jar.
>>
>> Also, what is the Flink version you are using?
>>
>> Cheers,
>> Gordon
>>
>

Mime
View raw message