zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Patrick Braga-Henebry <Patrick.Braga-Hene...@imc-chicago.com>
Subject zookeeper as configuration persistence medium?
Date Thu, 29 Nov 2012 00:52:47 GMT
Hi everyone,

We've been using ZooKeeper for service discovery for a couple months now and have been pretty

A bit about our setup: our ZK ensemble comprises 3 nodes and is spread across data centers.
 Compared to other ZK instances it is likely very low-traffic: perhaps 200 writes/1000 reads/500
clients per day.  So, small potatoes in terms of traffic.

Currently all nodes stored in our ZK are actively managed/populated by specific clients, such
that we would be able to theoretically wipe ZooKeeper and clients would repopulate znodes

I wanted to use the community as a sounding board here. Reading through the ZK literature
I've found, I haven't seen this use case mentioned explicitly:

Now we are considering storing configuration data in ZooKeeper.  Currently this configuration
is stored on the filesystem and read once at startup of a component.

My gut reaction is storing this data in ZooKeeper—using ZK itself as the persistence medium—is
a bad idea.  Because it would be storing data that isn't present anywhere else: supposing
a ZK crash, we would need to replay transaction logs to get the data back, rather than using
our existing backup solutions.

I think a better solution would be to write a component which is responsible for persisting
and managing updates to this configuration, and makes it available to interested clients via
ZooKeeper.  This would keep all znodes programmatically generated, which I think would lead
to a cleaner node space.  It would also help you avoid orphaned nodes, sitting around and
not being accessed/cleaned up anymore.  Eventually nobody would know what those orphaned nodes
are there for or if anyone is relying on their presence.

Am I on the right track here?  Are there ZK deployments out there being used in this way?
 In addition, is there a way to check the last time a znode was used?  Or, a pattern which
encourages not orphaning nodes in the first place?

Thanks in advance,
Patrick Braga-Henebry


The information in this e-mail is intended only for the person or entity to which it is addressed.

It may contain confidential and /or privileged material. If someone other than the intended
recipient should receive this e-mail, he / she shall not be entitled to read, disseminate,
disclose or duplicate it.

If you receive this e-mail unintentionally, please inform us immediately by "reply" and then
delete it from your system. Although this information has been compiled with great care, neither
IMC Financial Markets & Asset Management nor any of its related entities shall accept
any responsibility for any errors, omissions or other inaccuracies in this information or
for the consequences thereof, nor shall it be bound in any way by the contents of this e-mail
or its attachments. In the event of incomplete or incorrect transmission, please return the
e-mail to the sender and permanently delete this message and any attachments.

Messages and attachments are scanned for all known viruses. Always scan attachments before
opening them.

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message