kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Jay Kreps (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-554) Move all per-topic configuration into ZK and add to the CreateTopicCommand
Date Tue, 05 Mar 2013 01:04:13 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13592883#comment-13592883

Jay Kreps commented on KAFKA-554:

Yeah I thought about json vs properties. The advantage i am getting out of properties is the
ability to have non-stored defaults. Properties have the concept of defaults which are used
but not written out. I could probably replicate that with some wrapper class but that might
make the api a bit weird. The other option would be to use Properties in the api but just
serialize it as JSON. That is the option I am leaning towards.
> Move all per-topic configuration into ZK and add to the CreateTopicCommand
> --------------------------------------------------------------------------
>                 Key: KAFKA-554
>                 URL: https://issues.apache.org/jira/browse/KAFKA-554
>             Project: Kafka
>          Issue Type: New Feature
>            Reporter: Jay Kreps
>              Labels: project
>             Fix For: 0.8.1
>         Attachments: KAFKA-554-v1.patch
> We have a number of per-topic configurations that control message retention and flush
interval. Here is the list of properties I find in KafkaConfig that appear to be per-topic:
>   topic.log.file.size
>   topic.log.roll.hours
>   topic.log.retention.hours
>   topic.log.retention.size
>   topic.flush.intervals.ms
> Currently we specify these in server.properties. This is not a good solution as it requires
a rolling bounce of the cluster to make a change, which just doesn't scale to having hundreds
of topics. Also the map encoded in a CSV string is kind of hacky.
> We should move these into ZK in some kind of JSON blob that allows easily adding new
per-topic configs and we should remove these from server.properties.
> It would be good to start with a wiki design and get consensus on that first.

This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

View raw message