kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guozhang Wang (JIRA)" <j...@apache.org>
Subject [jira] [Created] (KAFKA-4587) Rethink Unification of Caching with Dedupping
Date Wed, 04 Jan 2017 06:54:58 GMT
Guozhang Wang created KAFKA-4587:
------------------------------------

             Summary: Rethink Unification of Caching with Dedupping
                 Key: KAFKA-4587
                 URL: https://issues.apache.org/jira/browse/KAFKA-4587
             Project: Kafka
          Issue Type: Bug
          Components: streams
            Reporter: Guozhang Wang


This is discussed in PR https://github.com/apache/kafka/pull/1588

In order to support user-customizable state store suppliers in the DSL we did the following:

1) Introduce a {{TupleForwarder}} to forward tuples from cached stores that is wrapping user
customized stores.
2) Narrow the scope to only dedup on forwarding if it is the default CachingXXStore with wrapper
RocksDB. 

With this, the unification of dedupping and caching is less useful now, and we are complicating
the inner implementations for forwarding a lot. We need to re-consider this feature with finer
granularity of turning on / off caching per store, potentially with explicit triggers.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message