kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Stephane Maarek (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-6323) punctuate with WALL_CLOCK_TIME triggered immediately
Date Fri, 08 Dec 2017 22:56:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6323?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16284366#comment-16284366

Stephane Maarek commented on KAFKA-6323:

Fully agree [~guozhang] . I fully agree on the punctuate once as well (even if T2 is 5 intervals
away), I have observed punctuate being called way to many times if the data does a big jump.

Finally, is there any interest or use cases in using both a wall clock and event driven punctuate?
Might require a KIP for that one

> punctuate with WALL_CLOCK_TIME triggered immediately
> ----------------------------------------------------
>                 Key: KAFKA-6323
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6323
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 1.0.0
>            Reporter: Frederic Arno
>            Assignee: Frederic Arno
>             Fix For: 1.1.0, 1.0.1
> When working on a custom Processor from which I am scheduling a punctuation using WALL_CLOCK_TIME.
I've noticed that whatever the punctuation interval I set, a call to my Punctuator is always
triggered immediately.
> Having a quick look at kafka-streams' code, I could find that all PunctuationSchedule's
timestamps are matched against the current time in order to decide whether or not to trigger
the punctuator (org.apache.kafka.streams.processor.internals.PunctuationQueue#mayPunctuate).
However, I've only seen code that initializes PunctuationSchedule's timestamp to 0, which
I guess is what is causing an immediate punctuation.
> At least when using WALL_CLOCK_TIME, shouldn't the PunctuationSchedule's timestamp be
initialized to current time + interval?

This message was sent by Atlassian JIRA

View raw message