kafka-jira mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From "Apurva Mehta (JIRA)" <j...@apache.org>
Subject [jira] [Commented] (KAFKA-5914) Return MessageFormatVersion and MessageMaxBytes in MetadataResponse
Date Tue, 19 Sep 2017 23:39:00 GMT

    [ https://issues.apache.org/jira/browse/KAFKA-5914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16172500#comment-16172500

Apurva Mehta commented on KAFKA-5914:

This is being descoped from the 1.0.0 release, since actually being able to return the message
format version and the message max bytes in a metadata request means that each broker must
have a consistent view of _all_ the topic configs in the cluster. 

This is harder to implement than originally thought since currently each broker knows only
about the topic configs of the topics actually present on the broker. So there is no global

Further, the update {{DynamicConfigManager}} update mechanism only propagates _updates_ to
the topic config, and doesn't notify brokers of topic creation.

So to maintain a global, consistent, cache of topic configs on each broker, we have to do
the following: 

# Pull the global topic config from zookeeper on startup. 
# Update the change notification path in zookeeper during topic creation so that the {{TopicConfigHandler}}
can be invoked and update the topic metadata cache on each broker.

> Return MessageFormatVersion and MessageMaxBytes in MetadataResponse
> -------------------------------------------------------------------
>                 Key: KAFKA-5914
>                 URL: https://issues.apache.org/jira/browse/KAFKA-5914
>             Project: Kafka
>          Issue Type: Sub-task
>            Reporter: Apurva Mehta
>            Assignee: Apurva Mehta
>             Fix For: 1.1.0
> As part of KIP-192, we want to send two additional fields in the {{TopicMetadata}} which
is part of the {{MetadataResponse}}. These fields are the {{MessageFormatVersion}} and the
> The {{MessageFormatVersion}} is required to implement https://issues.apache.org/jira/browse/KAFKA-5794
. The latter will be implemented in a future release, but with the changes proposed here,
the said future release will be backward compatible with 1.0.0

This message was sent by Atlassian JIRA

View raw message