flink-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Tzu-Li (Gordon) Tai (JIRA)" <j...@apache.org>
Subject [jira] [Created] (FLINK-6004) Allow FlinkKinesisConsumer to skip corrupted messages
Date Thu, 09 Mar 2017 06:21:37 GMT
Tzu-Li (Gordon) Tai created FLINK-6004:

             Summary: Allow FlinkKinesisConsumer to skip corrupted messages
                 Key: FLINK-6004
                 URL: https://issues.apache.org/jira/browse/FLINK-6004
             Project: Flink
          Issue Type: Improvement
            Reporter: Tzu-Li (Gordon) Tai

It is quite clear from the fix of FLINK-3679 that in reality, users might encounter corrupted
messages from Kafka / Kinesis / generally external sources when deserializing them.

The consumers should support simply skipping those messages, by letting the deserialization
schema return `null`, and checking `null` values within the consumer.

This has been done for the Kafka consumer already. This ticket tracks the improvement for
the Kinesis consumer.

This message was sent by Atlassian JIRA

View raw message