flink-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Yassine Marzougui <yassmar...@gmail.com>
Subject Re: Handling Kafka DeserializationSchema() exceptions
Date Thu, 25 Aug 2016 19:32:50 GMT
Good to know that there is already a JIRA issue, Thanks!

On Thu, Aug 25, 2016 at 8:58 PM, Robert Metzger <rmetzger@apache.org> wrote:

> Hi Yassine,
>
> there's an ongoing discussion about the issue in this JIRA:
> https://issues.apache.org/jira/browse/FLINK-3679.
> Emitting null is not an option.
> There are workarounds to the issue, but I think they are all not nice.
>
> On Thu, Aug 25, 2016 at 8:05 PM, Yassine Marzougui <yassmarzou@gmail.com>
> wrote:
>
>> Hi all,
>>
>> Is there a way to handle hafka deserialization exceptions, when a JSON
>> message is malformed for example?
>>
>> I thought about extending the DeserializationSchema to emit a null or any
>> other value, but that may cause an NPE when using a subsequent
>> TimestampExtractor.
>> The other solution would be to do the serialization in a subsequent
>> flatMap operator but it would be more convient to do it directly in the
>> consumer to make the timestamps extraction (with the new kafka consumer
>> timestamp extractor) easier.
>>
>> Any suggestions?
>>
>> Best,
>> Yassine
>>
>
>

Mime
View raw message