kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guozhang Wang (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-3777) Extract the LRU cache out of RocksDBStore
Date Fri, 03 Jun 2016 18:35:59 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15314561#comment-15314561
] 

Guozhang Wang commented on KAFKA-3777:
--------------------------------------

Adding to what Jay mentioned, we are already disabling the WAL logging of RocksDB in Kafka
Streams.

> Extract the LRU cache out of RocksDBStore
> -----------------------------------------
>
>                 Key: KAFKA-3777
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3777
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: streams
>    Affects Versions: 0.10.1.0
>            Reporter: Eno Thereska
>             Fix For: 0.10.1.0
>
>
> The LRU cache that is currently inside the RocksDbStore class. As part of KAFKA-3776
it needs to come outside of RocksDbStore and be a separate component used in:
> 1. KGroupedStream.aggregate() / reduce(), 
> 2. KStream.aggregateByKey() / reduceByKey(),
> 3. KTable.to() (this will be done in KAFKA-3779).
> As all of the above operators can have a cache on top to deduplicate the materialized
state store in RocksDB.
> The scope of this JIRA is to extract out the cache of RocksDBStore, and keep them as
item 1) and 2) above; and it should be done together / after KAFKA-3780.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message