zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Samoilov <samoi...@gmail.com>
Subject zookeeper watch limitation
Date Fri, 06 Jun 2014 21:36:46 GMT
hi,
I am reading the book "Zookeeper" by Flavio Junqueira and Benjamin Reed.
And I am now concerned if Zookeeper right tool for our scenario:
configuration management. We have ~2000 servers that expected to subscribe
to znode change notification: current version number. As version number
changed all clients will read new value and read configuration
correspoinding this value:

/ currentVersion "v3"
/versions
  /v1 {server1, server2, server3}
  /v2 {server1, server2, server5}
  /v3 {server0, server2, server3}

the idea we want to update configuration within seconds (<5s)

Is 2000 watch on same znode and than two simultaneous 2000 reads (one for
version and one for content) Ok for ZooKeeper?

according the book:
"...One issue to be aware of is that ZooKeeper triggers all watches set for
a particular znode change when the change occurs. If there are 1,000
clients that have set a watch on a given znode with a call to exists, then
1,000 notifications will be sent out when the znode is created. A change to
a watched znode might consequently generate a spike of notifica‚Äź tions.
Such a spike could affect, for example, the latency of operations submitted
around the time of the spike. When possible, we recommend avoiding such a
use of ZooKeeper in which a large number of clients watch for a change to a
given znode. It is much better to have only a few clients watching any
given znode at a time, and ideally at most one..."

1 vs 2000 is too big difference. And books says that even 1000 is a
problem. On other hand Zynga says that they did similar to our solution:
http://code.zynga.com/2011/08/updating-thousands-of-configuration-files-in-under-a-second/

Thank you,
Denis

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message