Mailing-List: contact kafka-dev-help@incubator.apache.org; run by ezmlm
Precedence: bulk
Reply-To: kafka-dev@incubator.apache.org
Date: Thu, 5 Jan 2012 21:38:39 +0000 (UTC)
From: "Jun Rao (Commented) (JIRA)" <jira@apache.org>
To: kafka-dev@incubator.apache.org
Message-ID: 
 <1885029879.11666.1325799519248.JavaMail.tomcat@hel.zones.apache.org>
In-Reply-To: 
 <1800535561.10805.1325788839341.JavaMail.tomcat@hel.zones.apache.org>
Subject: [jira] [Commented] (KAFKA-238) add a getTopicMetaData method in
 broker and expose it to producer
MIME-Version: 1.0
Content-Type: text/plain; charset=utf-8
Content-Transfer-Encoding: 7bit


    [ https://issues.apache.org/jira/browse/KAFKA-238?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13180855#comment-13180855 ] 

Jun Rao commented on KAFKA-238:
-------------------------------

I can see it's useful for the api to support a list of topics. Returning all topics is probably too much overhead since most producers are interested in a subset of topics.

In our current replication design, both send and fetch requests are served only by the leader. We can balance the load by having multiple partitions per broker. We may allow followers to serve read in the future. I don't see a clear benefit at this moment though. 
                
> add a getTopicMetaData method in broker and expose it to producer 
> ------------------------------------------------------------------
>
>                 Key: KAFKA-238
>                 URL: https://issues.apache.org/jira/browse/KAFKA-238
>             Project: Kafka
>          Issue Type: Sub-task
>          Components: core
>            Reporter: Jun Rao
>
> We need a way to propagate the leader and the partition information to the producer so that it can do load balancing and semantic partitioning. One way to do that is to have the producer get the information from ZK directly. This means that the producer needs to maintain a ZK session and has to subscribe to watchers, which can be complicated. An alternative approach is to have the following api on the broker.
> TopicMetaData getTopicMetaData(String: topic)
> TopicMetaData {
>   Array[PartitionMetaData]: partitionsMetaData
> }
> PartitionMetaData {
>   Int: partitionId
>   String: leaderHostname
>   Int: leaderPort
> }
> Using this api, the producer can get the metadata about a topic during initial startup or leadership change of a partition.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira