kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Khaireddine Rezgui (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-6535) Set default retention ms for Streams repartition topics to Long.MAX_VALUE
Date Thu, 12 Apr 2018 10:27:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16435307#comment-16435307

Khaireddine Rezgui commented on KAFKA-6535:

Hi guys, i updated theĀ [https://cwiki.apache.org/confluence/display/KAFKA/KIP-284%3A+Set+default+retention+ms+for+Streams+repartition+topics+to+Long.MAX_VALUE|KIP-284]
as accepted, can we proceed to the merge ?

> Set default retention ms for Streams repartition topics to Long.MAX_VALUE
> -------------------------------------------------------------------------
>                 Key: KAFKA-6535
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6535
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Guozhang Wang
>            Assignee: Khaireddine Rezgui
>            Priority: Major
>              Labels: needs-kip, newbie
> After KIP-220 / KIP-204, repartition topics in Streams are transient, so it is better
to set its default retention to infinity to allow any records be pushed to it with old timestamps
(think: bootstrapping, re-processing) and just rely on the purging API to keeping its storage
> More specifically, in {{RepartitionTopicConfig}} we have a few default overrides for
repartition topic configs, we should just add the override for {{TopicConfig.RETENTION_MS_CONFIG}}
to set it to Long.MAX_VALUE. This still allows users to override themselves if they want via
{{StreamsConfig.TOPIC_PREFIX}}. We need to add unit test to verify this update takes effect.
> In addition to the code change, we also need to have doc changes in streams/upgrade_guide.html
specifying this default value change.

This message was sent by Atlassian JIRA

View raw message