kafka-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Chris Riccomini <criccom...@linkedin.com>
Subject Re: About kafka 0.8 producer zookeeper-based load balancing on per-request basis
Date Tue, 15 Jan 2013 19:25:32 GMT
Hey Guys,

Correct me if I'm wrong, but I believe in 0.8, the producer uses a
metadata request to get topic/partition mappings from the broker. The
broker then interacts with ZK (rather than having the producer do it using

In the event that the master for a topic/partition fails, a new master
broker will be elected for a given topic/partition pair. When the producer
tries to send to the old broker (which is either dead, or a slave now),
the broker will either not respond, or the response will contain an error
code. In either case, I think the producer will do a new metadata request
(to a broker) to get the latest topic/partition to broker mappings. In
this way, it avoids having to use ZooKeeper: it offloads all ZK work to
the broker.


On 1/14/13 9:35 PM, "Jun Guo -X (jungu - CIIC at Cisco)" <jungu@cisco.com>

>We know, in kafka 0.8, producer connect to broker directly, it without
>connecting to zookeeper. Than how it achieve zookeeper-based load balance
>on per-request basis?
>Actually, when a topic be created, its partition will distributed in one
>or more brokers. When a message be sent, it will be delivered to a
>certain partition according to its key word. That is to say ,a certain
>must be sent to a fixed partition on a fixed broker. How the so called
>load balancing works?
>Best Regards

View raw message