kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "John Roesler (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-6535) Set default retention ms for Streams repartition topics to Long.MAX_VALUE
Date Thu, 22 Mar 2018 22:34:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16410509#comment-16410509

John Roesler commented on KAFKA-6535:

It seems unlikely that there might be folks who just use the default and are adversely affected
by the new default, so I'd "vote" not to bother with a KIP. But I'm also new in town...

> Set default retention ms for Streams repartition topics to Long.MAX_VALUE
> -------------------------------------------------------------------------
>                 Key: KAFKA-6535
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6535
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Guozhang Wang
>            Assignee: Khaireddine Rezgui
>            Priority: Major
>              Labels: needs-kip, newbie
> After KIP-220 / KIP-204, repartition topics in Streams are transient, so it is better
to set its default retention to infinity to allow any records be pushed to it with old timestamps
(think: bootstrapping, re-processing) and just rely on the purging API to keeping its storage
> More specifically, in {{RepartitionTopicConfig}} we have a few default overrides for
repartition topic configs, we should just add the override for {{TopicConfig.RETENTION_MS_CONFIG}}
to set it to Long.MAX_VALUE. This still allows users to override themselves if they want via
{{StreamsConfig.TOPIC_PREFIX}}. We need to add unit test to verify this update takes effect.
> In addition to the code change, we also need to have doc changes in streams/upgrade_guide.html
specifying this default value change.

This message was sent by Atlassian JIRA

View raw message