zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Benjamin Reed <br...@apache.org>
Subject Re: zookeeper watch limitation
Date Sat, 07 Jun 2014 15:06:21 GMT
in the section you are referring to we are pointing out that sometimes you
can reduce the load on ZooKeeper if you avoid waking up the herd when you
don't have to. waking up 1000 processes so that one of them can become
leader is an unnecessary cost, but if you want everyone to know about a
change then it is perfectly ok. you will see a spike in load, but it is a
needed spike. 2000 simultaneous reads is really not a problem at all. our
initial design point (years ago) was 10,000 in a second, but we have far
exceeded that performance goal.

ben


On Fri, Jun 6, 2014 at 2:36 PM, Denis Samoilov <samoilov@gmail.com> wrote:

> hi,
> I am reading the book "Zookeeper" by Flavio Junqueira and Benjamin Reed.
> And I am now concerned if Zookeeper right tool for our scenario:
> configuration management. We have ~2000 servers that expected to subscribe
> to znode change notification: current version number. As version number
> changed all clients will read new value and read configuration
> correspoinding this value:
>
> / currentVersion "v3"
> /versions
>   /v1 {server1, server2, server3}
>   /v2 {server1, server2, server5}
>   /v3 {server0, server2, server3}
>
> the idea we want to update configuration within seconds (<5s)
>
> Is 2000 watch on same znode and than two simultaneous 2000 reads (one for
> version and one for content) Ok for ZooKeeper?
>
> according the book:
> "...One issue to be aware of is that ZooKeeper triggers all watches set for
> a particular znode change when the change occurs. If there are 1,000
> clients that have set a watch on a given znode with a call to exists, then
> 1,000 notifications will be sent out when the znode is created. A change to
> a watched znode might consequently generate a spike of notifica‚Äź tions.
> Such a spike could affect, for example, the latency of operations submitted
> around the time of the spike. When possible, we recommend avoiding such a
> use of ZooKeeper in which a large number of clients watch for a change to a
> given znode. It is much better to have only a few clients watching any
> given znode at a time, and ideally at most one..."
>
> 1 vs 2000 is too big difference. And books says that even 1000 is a
> problem. On other hand Zynga says that they did similar to our solution:
>
> http://code.zynga.com/2011/08/updating-thousands-of-configuration-files-in-under-a-second/
>
> Thank you,
> Denis
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message