zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Mahadev Konar <maha...@yahoo-inc.com>
Subject Re: Multiple ZK clusters or a single, shared cluster?
Date Fri, 17 Jul 2009 20:34:20 GMT
Hi Jonathan,
  Regarding sharing ZooKeeper servers among applications, we actually
recommend doing that. We ask our users to have dedicated 3-5 node clusters
with dedicated machines(/disk) for zookeeper and lets applications share
those 3-5 nodes clusters.

But we do have have users (inside Yahoo!) wherein they want to run these
clusters for themselves. The reasons being:

- there usage of zookeeper (in terms of reads/writes per second and number
of clients - 5K or so) is pretty high and they do not want to share the
dedicated cluster with other properties inside Yahoo!

- they also want to manage upgrades/administration of cluster on there own

There might be more info on administering your own cluster at


On 7/17/09 12:48 PM, "Jonathan Gray" <jgray@apache.org> wrote:

> Hey guys,
> Been using ZK indirectly for a few months now in the HBase and Katta
> realms.  Both of these applications make it really easy so you don't
> have to be involved much with managing your ZK cluster to support it.
> I'm now using ZK for a bunch of things internally, so now I'm manually
> configuring, starting, and managing a cluster.
> What advice is there about whether I should be sharing a single cluster
> between all my applications, or running separate ones for each use?
> I've been told that it's strongly recommended to run your ZK nodes
> separately from the application using them (this is actually what we're
> telling new users over in HBase, though a majority of installations will
> likely co-host them with DataNodes and RegionServers).
> I don't have the resources to maintain a separate 3+ node ZK cluster for
> each of my applications, so this is not really an option.  I'm trying to
> decide if I should have HBase running/managing it's own ZK cluster that
> is co-located with some of the regionservers (there will be ample
> memory, but ZK will not have a dedicated disk), or if I should be
> pointing it to a dedicated 3 node ZK cluster.
> I would then also have Katta pointing at this same shared cluster (or a
> separate cluster would be co-located with katta nodes).  Same for my
> application; could share nodes with the app servers or pointed at a
> single ZK cluster.
> Trade-offs I should be aware of?  Current best practices?
> Any help would be much appreciated.  Thanks.
> Jonathan Gray

View raw message