kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Bill Bejeck (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-6248) Enable configuration of internal topics of Kafka Streams applications
Date Tue, 21 Nov 2017 15:24:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-6248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16260892#comment-16260892

Bill Bejeck commented on KAFKA-6248:

Hi [~timvanlaer] 

Take a look at [KIP-173|https://cwiki.apache.org/confluence/display/KAFKA/KIP-173%3A+Add+prefix+to+StreamsConfig+to+enable+setting+default+internal+topic+configs.]
included in  Kafka Streams as of release 1.0, which allows you to configure internal topics
using the {{topic.}} prefix.  

Note this is a global setting for all internal topics and the config needs to be a valid property
from {{org.apache.kafka.common.config.TopicConfig}}


> Enable configuration of internal topics of Kafka Streams applications
> ---------------------------------------------------------------------
>                 Key: KAFKA-6248
>                 URL: https://issues.apache.org/jira/browse/KAFKA-6248
>             Project: Kafka
>          Issue Type: Improvement
>          Components: streams
>            Reporter: Tim Van Laer
>            Priority: Minor
> In the current implementation of Kafka Streams, it is not possible to set custom configuration
to internal topics (e.g. max.message.bytes, retention.ms...). It would be nice if a developer
can set some specific configuration. 
> E.g. if you want to store messages bigger than 1MiB in a state store, you have to alter
the corresponding changelog topic with a max.message.bytes setting. 
> The workaround is to create the 'internal' topics upfront using the correct naming convention
so Kafka Streams will use the explicitly defined topics as if they are internal. 
> An alternative is to alter the internal topics after the Kafka Streams application is
started and has created its internal topics. 

This message was sent by Atlassian JIRA

View raw message