kafka-dev mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Jay Kreps <jay.kr...@gmail.com>
Subject zookeeper partition data
Date Thu, 17 Jan 2013 03:47:49 GMT
Why is the node<=>partition mapping stored as strings?

get /brokers/topics/my-topic
{ "0": ["0"] }

This is a bug, right? Partitions and node ids are ints.

Also, this structure is not very extensible. If you add anything to this
map that is not a partition-node mapping it would break. I think the way
this should have been done is

{
  "partitions:{0:[1,3,2], 1:[3,1,7], ...}
}

Then we could add more attributes to the map:

{
  "partitions:{0:[1,3,2], 1:[3,1,7], ...}
  "settings":{
     "log.segment.bytes":1234234,
      ...
   }
}

I think our zookeeper data structures are another "contract" it would be
good to document and review these before the 0.8 release. If we want to
change something now it is really easy but once it is out there all changes
will need to be backwards compatible since they are persistent and can be
shared by all brokers (so rolling upgrade will be difficult).

-Jay

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message