zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Manikumar Reddy <ku...@nmsworks.co.in>
Subject Re: Zookeeper use cases with Kafka
Date Tue, 18 Aug 2015 13:36:28 GMT
Hi,

 1.  ZK is used for  co-ordination between brokers, controller election,
leader election,  storing topic configuration etc.
      I think we use both sequential and ephemeral nodes.

 2.  Yes, Kafka uses ZK watches for controller changes, new topic creation,
new partition creation, leader change on partition etc.  some links here:


https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/consumer/ZookeeperTopicEventWatcher.scala

https://github.com/apache/kafka/blob/trunk/core/src/main/scala/kafka/controller/KafkaController.scala

 3. Kafka data structures in ZK are documented here:

https://cwiki.apache.org/confluence/display/KAFKA/Kafka+data+structures+in+Zookeeper


Kumar

On Tue, Aug 18, 2015 at 6:33 PM, Prabhjot Bharaj <prabhbharaj@gmail.com>
wrote:

> Hello Folks,
>
> I wish to contribute to Kafka internals. And, one of the things which can
> help me do that is understanding how kafka uses zookeeper. I have some of
> these basic doubts:-
>
> 1. Is zookeeper primarily used for locking ? If yes, in what cases and what
> kind of nodes does it use - sequential/ephemeral?
>
> 2. Does kafka use zookeeper watches for any of functions ?
>
> 3. What kind of state is stored in Zookeeper ? (I believe it has to be the
> leader information per partition, but is there anything apart from it?)
> What is the scale of data that is stored in Zookeeper ?
>
> Looking forward for your help.
>
> Thanks,
> prabcs
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message