kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax (Jira)" <j...@apache.org>
Subject [jira] [Resolved] (KAFKA-4729) Stores for kstream-kstream join cannot be in-memory
Date Thu, 05 Sep 2019 23:20:00 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-4729?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Matthias J. Sax resolved KAFKA-4729.
    Resolution: Duplicate

> Stores for kstream-kstream join cannot be in-memory
> ---------------------------------------------------
>                 Key: KAFKA-4729
>                 URL: https://issues.apache.org/jira/browse/KAFKA-4729
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>    Affects Versions:
>            Reporter: Eno Thereska
>            Priority: Major
> Whereas we can specify in the DSL that stores used for aggregates can be RocksDb-based
on in-memory, we cannot do that for stores used for KStream-KStream joins. E.g., the join()
methon in KStreamImpl.java creates two state stores and the user does not have the option
of having them be in-memory:
> StateStoreSupplier thisWindow =
>                 createWindowedStateStore(windows, keySerde, lhsValueSerde, joinThisName
+ "-store");
>             StateStoreSupplier otherWindow =
>                 createWindowedStateStore(windows, keySerde, otherValueSerde, joinOtherName
+ "-store");
> Part of the problem is that for joins, stores are not exposed to the user. We might want
to rethink that.

This message was sent by Atlassian Jira

View raw message