kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Matthias J. Sax (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-3060) Refactor MeteredXXStore
Date Wed, 22 Feb 2017 01:01:50 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-3060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel

Matthias J. Sax updated KAFKA-3060:
    Affects Version/s:     (was:

> Refactor MeteredXXStore
> -----------------------
>                 Key: KAFKA-3060
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3060
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: streams
>            Reporter: Yasuhiro Matsuda
>            Assignee: Guozhang Wang
>            Priority: Minor
>             Fix For:
> ** copied from a github comment by Guozhang Wang **
> The original motivation of having the MeteredXXStore is to wrap all metrics / logging
semantics into one place so they do not need to be re-implemented again, but this seems to
be an obstacle with the current pattern now, for example MeteredWindowStore.putAndReturnInternalKey
is only used for logging, and MeteredWindowStore.putInternal / MeteredWindowStore.getInternal
are never used since only its inner will trigger this function. So how about refactoring this
piece as follows:
> 1. WindowStore only expose two APIs: put(K, V) and get(K, long).
> 2. Add a RollingRocksDBStores that does not extend any interface, but only implements
putInternal, getInternal and putAndReturnInternalKey that uses underlying RocksDBStore<byte[],
byte[]> as Segments.
> 3. RocksDBWindowStore implements WindowStore with an RollingRocksDBStores inner.
> 4. Let MeteredXXStore only maintain the metrics recording logic, and let different stores
implement their own logging logic, since this is now different across different types and
are better handled separately. Also some types of stores may not even have a loggingEnabled
flag, if it will always log, or will never log.

This message was sent by Atlassian JIRA

View raw message