kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Evgeny Veretennikov (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-4468) Correctly calculate the window end timestamp after read from state stores
Date Thu, 06 Jul 2017 11:18:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-4468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16076373#comment-16076373

Evgeny Veretennikov commented on KAFKA-4468:

Sorry, guys, I missed javadoc comment to {{WindowedDeserializer}}... :)

The inner deserializer class can be specified by setting the property key.deserializer.inner.class,
value.deserializer.inner.class or deserializer.inner.class

So, users actually can use this class via property.

By the way, why Kafka users are allowed to use internal Kafka streams classes? Doesn't it
make sense to move such classes, which users can use, outside of {{internals}} package?

> Correctly calculate the window end timestamp after read from state stores
> -------------------------------------------------------------------------
>                 Key: KAFKA-4468
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4468
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>            Reporter: Guozhang Wang
>              Labels: architecture
> When storing the WindowedStore on the persistent KV store, we only use the start timestamp
of the window as part of the combo-key as (start-timestamp, key). The reason that we do not
add the end-timestamp as well is that we can always calculate it from the start timestamp
+ window_length, and hence we can save 8 bytes per key on the persistent KV store.
> However, after read it (via {{WindowedDeserializer}}) we do not set its end timestamp
correctly but just read it as an {{UnlimitedWindow}}. We should fix this by calculating its
end timestamp as mentioned above.

This message was sent by Atlassian JIRA

View raw message