zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: zookeeper watch limitation
Date Sat, 07 Jun 2014 02:46:01 GMT
It is a problem if you expect subsequent watches to go out in milliseconds.

It isn't a problem if the resulting delays are OK with you.  To me, it
sounds like it will be just fine.  If the herd effect is too much, you can
always split the version flags into many pieces and update one version flag
at a time setting of a small herd each time.  That would also allow you to
do canary testing with new configs.




On Fri, Jun 6, 2014 at 2:36 PM, Denis Samoilov <samoilov@gmail.com> wrote:

> hi,
> I am reading the book "Zookeeper" by Flavio Junqueira and Benjamin Reed.
> And I am now concerned if Zookeeper right tool for our scenario:
> configuration management. We have ~2000 servers that expected to subscribe
> to znode change notification: current version number. As version number
> changed all clients will read new value and read configuration
> correspoinding this value:
>
> / currentVersion "v3"
> /versions
>   /v1 {server1, server2, server3}
>   /v2 {server1, server2, server5}
>   /v3 {server0, server2, server3}
>
> the idea we want to update configuration within seconds (<5s)
>
> Is 2000 watch on same znode and than two simultaneous 2000 reads (one for
> version and one for content) Ok for ZooKeeper?
>
> according the book:
> "...One issue to be aware of is that ZooKeeper triggers all watches set for
> a particular znode change when the change occurs. If there are 1,000
> clients that have set a watch on a given znode with a call to exists, then
> 1,000 notifications will be sent out when the znode is created. A change to
> a watched znode might consequently generate a spike of notifica‚Äź tions.
> Such a spike could affect, for example, the latency of operations submitted
> around the time of the spike. When possible, we recommend avoiding such a
> use of ZooKeeper in which a large number of clients watch for a change to a
> given znode. It is much better to have only a few clients watching any
> given znode at a time, and ideally at most one..."
>
> 1 vs 2000 is too big difference. And books says that even 1000 is a
> problem. On other hand Zynga says that they did similar to our solution:
>
> http://code.zynga.com/2011/08/updating-thousands-of-configuration-files-in-under-a-second/
>
> Thank you,
> Denis
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message