kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Cemalettin Koç (JIRA) <j...@apache.org>
Subject [jira] [Comment Edited] (KAFKA-6711) GlobalStateManagerImpl should not write offsets of in-memory stores in checkpoint file
Date Tue, 27 Mar 2018 07:14:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16414673#comment-16414673
] 

Cemalettin Koç edited comment on KAFKA-6711 at 3/27/18 7:13 AM:
----------------------------------------------------------------

[~guozhang] Would you please check implementation please: 

https://github.com/cemo/kafka/commits/b-kafka-6711

I have done something preliminary. I will add necessary tests as well after your guidance.


was (Author: cemo):
[~guozhang] Would you please check implementation please: 

[https://github.com/cemo/kafka/commit/0cb2482259fec897f396e8b84ffb1921c4f3f63e]

I have done something preliminary. I will add necessary tests as well after your guidance.

> GlobalStateManagerImpl should not write offsets of in-memory stores in checkpoint file
> --------------------------------------------------------------------------------------
>
>                 Key: KAFKA-6711
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6711
>             Project: Kafka
>          Issue Type: Bug
>          Components: streams
>    Affects Versions: 1.0.1
>            Reporter: Cemalettin Koç
>            Assignee: Cemalettin Koç
>            Priority: Major
>              Labels: newbie
>
> We are using an InMemoryStore along with GlobalKTable and I noticed that after each restart
I am losing all my data. When I debug it, `/tmp/kafka-streams/category-client-1/global/.checkpoint`
file contains offset for my GlobalKTable topic. I had checked GlobalStateManagerImpl implementation
and noticed that it is not guarded for cases similar to mine. I am fairly new to Kafka land
and probably there might be another way to fix issue. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Mime
View raw message