zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: Multiple ZK clusters or a single, shared cluster?
Date Fri, 17 Jul 2009 22:50:45 GMT
This may be true for monster clusters being supported by a ZK cluster or for
data-intensive operations like high throughput queuing, but many
applications of ZK are incredibly low volume.

For instance, with Katta, there is a read-modify-write when a search engine
comes up, when a shard is assigned or loaded, a dozen or so when an index is
added and a few when the master fails.  None of these operations occurs in
practice even once an hour.

I also log machine state across lots of machines once every few seconds
(partly to give ZK something to do).  That results in dozens (at most) of
transactions per second.

I think that a huge proportion of Zk clusters are going to have transactions
per day rather than kilo transactions per second.  At these rates, all you
have to worry about is simultaneous failures ... make sure that the ZK
cluster members are on separate power supplies and network switches.  Beyond
that, everything is over-kill.  You could run on a cluster of iPhones and
still be happy with performance.

On Fri, Jul 17, 2009 at 1:38 PM, Benjamin Reed <breed@yahoo-inc.com> wrote:

> the main thing is that you use dedicated zk machines (with a dedicated disk
> for logging). once you have that in place, watch the load on your cluster,
> as long as you aren't saturating the cluster you should share.

Ted Dunning, CTO

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message