kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Guozhang Wang (JIRA)" <j...@apache.org>
Subject [jira] [Updated] (KAFKA-3184) Add Checkpoint for In-memory State Store
Date Fri, 23 Sep 2016 22:56:20 GMT

     [ https://issues.apache.org/jira/browse/KAFKA-3184?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]

Guozhang Wang updated KAFKA-3184:
---------------------------------
    Fix Version/s:     (was: 0.10.1.0)
                   0.10.2.0

> Add Checkpoint for In-memory State Store
> ----------------------------------------
>
>                 Key: KAFKA-3184
>                 URL: https://issues.apache.org/jira/browse/KAFKA-3184
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Guozhang Wang
>              Labels: user-experience
>             Fix For: 0.10.2.0
>
>
> Currently Kafka Streams does not make a checkpoint of the persistent state store upon
committing, which would be expensive since it is "stopping the world" and write on disks:
for example, RocksDB would require you to copy the file directory to make a copy naively.

> However, for in-memory stores checkpointing maybe doable in an asynchronous manner hence
it can be done quickly. And the benefit of having intermediate checkpoint is to avoid restoring
from scratch if standby tasks are not present.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

Mime
View raw message