incubator-kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jun Rao <jun...@gmail.com>
Subject Re: ZooKeeper-based Producer behavior before there are topics
Date Tue, 06 Dec 2011 17:02:48 GMT
David,

There is some logic in ZKBrokerPartitionInfo.bootstrapWithExistingBrokers
that deals with the case that the topic is initially empty.

Currently, there is no API to get metadata. One has to query ZK directly.
We can discuss whether to add metadata apis. Do you want to create a jira
for that?

Thanks,

Jun

On Tue, Dec 6, 2011 at 6:34 AM, David Ormsbee <dave@datadoghq.com> wrote:

> Hi folks,
>
> I'm implementing ZooKeeper support in our Python client, and I was
> wondering what the expected behavior of the Producer is when the brokers
> are completely empty. At this point, the brokers have registered themselves
> in /brokers/ids/[0..N], but there are no topics registered under
> /brokers/topics. If someone wants to create a Producer at this point and
> start sending messages to a new topic, is that an error because no brokers
> are registered for that topic? Or should I just make the initial assumption
> that all brokers have one partition, send messages to those to create the
> topic on the brokers, and then readjust when the the brokers publish their
> partitions-per-topic information to ZK after the initial writes?
>
> I'm sure I'm missing something obvious here, since we'd run into a similar
> problem for new, empty brokers joining the pool.
>
> As an aside, is there any way to query metadata about the broker directly?
> So that a simple consumer might be able to ask a broker for what topics
> they have, how many partitions, what version it's running, etc? Or is that
> only ever published through ZooKeeper?
>
> Thank you.
>
> Dave
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message