kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Nikki Thean (JIRA)" <j...@apache.org>
Subject [jira] [Comment Edited] (KAFKA-5055) Kafka Streams skipped-records-rate sensor producing nonzero values even when FailOnInvalidTimestamp is used as extractor
Date Fri, 21 Apr 2017 21:39:04 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15979433#comment-15979433
] 

Nikki Thean edited comment on KAFKA-5055 at 4/21/17 9:38 PM:
-------------------------------------------------------------

[~guozhang] Sure. Streams config properties were [https://gist.github.com/nixsticks/ccbfdbb6141e0f7362cef53d9e39d39b].


The nonzero value -- it depends. I have a topic that receives billions of records per day,
not sure how many per second. The nonzero value tends to be very low when the lag is high
(when I start reading the topic at earliest offset), and then rises to about 400 once I am
consuming recent records.


was (Author: nthean):
[~guozhang] Sure. Streams config properties were (these)[https://gist.github.com/nixsticks/ccbfdbb6141e0f7362cef53d9e39d39b].
The nonzero value -- it depends. I have a topic that receives billions of records per day,
not sure how many per second. The nonzero value tends to be very low when the lag is high
(when I start reading the topic at earliest offset), and then rises to about 400 once I am
consuming recent records.

> Kafka Streams skipped-records-rate sensor producing nonzero values even when FailOnInvalidTimestamp
is used as extractor
> ------------------------------------------------------------------------------------------------------------------------
>
>                 Key: KAFKA-5055
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5055
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>            Reporter: Nikki Thean
>
> According to the code and the documentation for this metric, the only reason for a skipped
record is an invalid timestamp, except that a) I am reading from a topic that is populated
solely by Kafka Connect and b) I am using `FailOnInvalidTimestamp` as the timestamp extractor.
> Either I'm missing something in the documentation (i.e. another reason for skipped records)
or there is a bug in the code that calculates this metric.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Mime
View raw message