kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Eno Thereska (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-3777) Extract the existing LRU cache out of RocksDBStore
Date Mon, 18 Jul 2016 22:59:20 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-3777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Eno Thereska updated KAFKA-3777:
--------------------------------
    Description: 
The LRU cache that is currently inside the RocksDbStore class. As part of KAFKA-3776 it needs
to come outside of RocksDbStore and be a separate component used in:

1. KGroupedStream.aggregate() / reduce(), 
2. KStream.aggregateByKey() / reduceByKey(),
3. KTable.to() (this will be done in KAFKA-3779).

As all of the above operators can have a cache on top to deduplicate the materialized state
store in RocksDB.

The scope of this JIRA is to extract out the cache of RocksDBStore, and keep them as item
1) and 2) above; and it should be done together / after KAFKA-3780.

Note it is NOT in the scope of this JIRA to re-write the cache, so this will basically stay
the same record-based cache we currently have.

  was:
The LRU cache that is currently inside the RocksDbStore class. As part of KAFKA-3776 it needs
to come outside of RocksDbStore and be a separate component used in:

1. KGroupedStream.aggregate() / reduce(), 
2. KStream.aggregateByKey() / reduceByKey(),
3. KTable.to() (this will be done in KAFKA-3779).

As all of the above operators can have a cache on top to deduplicate the materialized state
store in RocksDB.

The scope of this JIRA is to extract out the cache of RocksDBStore, and keep them as item
1) and 2) above; and it should be done together / after KAFKA-3780.


> Extract the existing LRU cache out of RocksDBStore
> --------------------------------------------------
>
>                 Key: KAFKA-3777
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3777
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: streams
>    Affects Versions: 0.10.1.0
>            Reporter: Eno Thereska
>            Assignee: Anna Povzner
>             Fix For: 0.10.1.0
>
>
> The LRU cache that is currently inside the RocksDbStore class. As part of KAFKA-3776
it needs to come outside of RocksDbStore and be a separate component used in:
> 1. KGroupedStream.aggregate() / reduce(), 
> 2. KStream.aggregateByKey() / reduceByKey(),
> 3. KTable.to() (this will be done in KAFKA-3779).
> As all of the above operators can have a cache on top to deduplicate the materialized
state store in RocksDB.
> The scope of this JIRA is to extract out the cache of RocksDBStore, and keep them as
item 1) and 2) above; and it should be done together / after KAFKA-3780.
> Note it is NOT in the scope of this JIRA to re-write the cache, so this will basically
stay the same record-based cache we currently have.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message