kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michał (Jira) <j...@apache.org>
Subject [jira] [Created] (KAFKA-8924) Default grace period (-1) of TimeWindows causes suppress to never emit events
Date Thu, 19 Sep 2019 10:30:00 GMT
Michał created KAFKA-8924:

             Summary: Default grace period (-1) of TimeWindows causes suppress to never emit
                 Key: KAFKA-8924
                 URL: https://issues.apache.org/jira/browse/KAFKA-8924
             Project: Kafka
          Issue Type: Bug
          Components: streams
    Affects Versions: 2.3.0
            Reporter: Michał


The default creation of TimeWindows, like


calls an internal constructor

return new TimeWindows(sizeMs, sizeMs, -1, DEFAULT_RETENTION_MS);

And the *-1* parameter is the default grace period which I think is here for backward compatibility

    @SuppressWarnings("deprecation") // continuing to support Windows#maintainMs/segmentInterval
in fallback mode
    public long gracePeriodMs() {
        // NOTE: in the future, when we remove maintainMs,
        // we should default the grace period to 24h to maintain the default behavior,
        // or we can default to (24h - size) if you want to be super accurate.
        return graceMs != -1 ? graceMs : maintainMs() - size();

The problem is that if you use a TimeWindows with gracePeriod of *-1* together with suppress
*untilWindowCloses*, it never emits an event.

You can check the Suppress tests (SuppressScenarioTest.shouldSupportFinalResultsForTimeWindows),
where [~vvcephei] was (maybe) aware of that and all the scenarios specify the gracePeriod.

I will add a test without it on my branch and it will fail.

#Now what can be done

One easy fix would be to change the default value to 0, which works fine for me in my project,
however, I am not aware of the impact it would have done due to the changes in the *gracePeriodMs*
method mentioned before.

This message was sent by Atlassian Jira

View raw message