kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Michał (Jira) <j...@apache.org>
Subject [jira] [Created] (KAFKA-8924) Default grace period (-1) of TimeWindows causes suppress to never emit events
Date Thu, 19 Sep 2019 10:30:00 GMT
Michał created KAFKA-8924:
-----------------------------

             Summary: Default grace period (-1) of TimeWindows causes suppress to never emit
events 
                 Key: KAFKA-8924
                 URL: https://issues.apache.org/jira/browse/KAFKA-8924
             Project: Kafka
          Issue Type: Bug
          Components: streams
    Affects Versions: 2.3.0
            Reporter: Michał


#Problem 

The default creation of TimeWindows, like

```
TimeWindows.of(ofMillis(xxx))
```

calls an internal constructor

```
return new TimeWindows(sizeMs, sizeMs, -1, DEFAULT_RETENTION_MS);
```

And the *-1* parameter is the default grace period which I think is here for backward compatibility

```
    @SuppressWarnings("deprecation") // continuing to support Windows#maintainMs/segmentInterval
in fallback mode
    @Override
    public long gracePeriodMs() {
        // NOTE: in the future, when we remove maintainMs,
        // we should default the grace period to 24h to maintain the default behavior,
        // or we can default to (24h - size) if you want to be super accurate.
        return graceMs != -1 ? graceMs : maintainMs() - size();
    }
```

The problem is that if you use a TimeWindows with gracePeriod of *-1* together with suppress
*untilWindowCloses*, it never emits an event.

You can check the Suppress tests (SuppressScenarioTest.shouldSupportFinalResultsForTimeWindows),
where [~vvcephei] was (maybe) aware of that and all the scenarios specify the gracePeriod.

I will add a test without it on my branch and it will fail.

#Now what can be done

One easy fix would be to change the default value to 0, which works fine for me in my project,
however, I am not aware of the impact it would have done due to the changes in the *gracePeriodMs*
method mentioned before.





--
This message was sent by Atlassian Jira
(v8.3.4#803005)

Mime
View raw message