kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Ewen Cheslack-Postava (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-4393) Improve invalid/negative TS handling
Date Mon, 02 Jan 2017 20:45:58 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-4393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Ewen Cheslack-Postava updated KAFKA-4393:
    Labels: kip  (was: )

> Improve invalid/negative TS handling
> ------------------------------------
>                 Key: KAFKA-4393
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4393
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Matthias J. Sax
>            Assignee: Matthias J. Sax
>              Labels: kip
>             Fix For:
> Currently, Kafka Streams does not handle invalid/negative timestamps returned from the
{{TimestampExtractor}} gracefully, but fails with an exception, because negative timestamps
cannot get handled in a meaningful way for any time based (ie, window) operators like window
aggregates and joins.
> We want to change Streams to a auto-drop behavior for negative timestamps for those records
(without any further user notification about dropped record) to enable users to "step over"
those records and keep going (instead of an exception). To guard the user from silently dropping
messages by default (and kept current fail-fast behavior), we change the default extractor
{{ConsumerRecordTimestampExtractor}} to check the extracted meta-data record timestamp and
raise an exception if it is negative. Furthermore, we add a "drop-and-log" extractor, as this
seems to be a common behavior user might want to have. For any other behavior, users can still
provide a custom TS-Extractor implementation.

This message was sent by Atlassian JIRA

View raw message