kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jay Kreps (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-554) Move all per-topic configuration into ZK and add to the CreateTopicCommand
Date Tue, 05 Mar 2013 01:04:13 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13592883#comment-13592883
] 

Jay Kreps commented on KAFKA-554:
---------------------------------

Yeah I thought about json vs properties. The advantage i am getting out of properties is the
ability to have non-stored defaults. Properties have the concept of defaults which are used
but not written out. I could probably replicate that with some wrapper class but that might
make the api a bit weird. The other option would be to use Properties in the api but just
serialize it as JSON. That is the option I am leaning towards.
                
> Move all per-topic configuration into ZK and add to the CreateTopicCommand
> --------------------------------------------------------------------------
>
>                 Key: KAFKA-554
>                 URL: https://issues.apache.org/jira/browse/KAFKA-554
>             Project: Kafka
>          Issue Type: New Feature
>            Reporter: Jay Kreps
>              Labels: project
>             Fix For: 0.8.1
>
>         Attachments: KAFKA-554-v1.patch
>
>
> We have a number of per-topic configurations that control message retention and flush
interval. Here is the list of properties I find in KafkaConfig that appear to be per-topic:
>   topic.log.file.size
>   topic.log.roll.hours
>   topic.log.retention.hours
>   topic.log.retention.size
>   topic.flush.intervals.ms
> Currently we specify these in server.properties. This is not a good solution as it requires
a rolling bounce of the cluster to make a change, which just doesn't scale to having hundreds
of topics. Also the map encoded in a CSV string is kind of hacky.
> We should move these into ZK in some kind of JSON blob that allows easily adding new
per-topic configs and we should remove these from server.properties.
> It would be good to start with a wiki design and get consensus on that first.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

Mime
View raw message