zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Denis Samoilov <samoi...@gmail.com>
Subject Re: zookeeper watch limitation
Date Tue, 10 Jun 2014 03:51:15 GMT
Ben, Ted,
thank you very much for the answers!


On Sat, Jun 7, 2014 at 8:06 AM, Benjamin Reed <breed@apache.org> wrote:

> in the section you are referring to we are pointing out that sometimes you
> can reduce the load on ZooKeeper if you avoid waking up the herd when you
> don't have to. waking up 1000 processes so that one of them can become
> leader is an unnecessary cost, but if you want everyone to know about a
> change then it is perfectly ok. you will see a spike in load, but it is a
> needed spike. 2000 simultaneous reads is really not a problem at all. our
> initial design point (years ago) was 10,000 in a second, but we have far
> exceeded that performance goal.
>
> ben
>
>
> On Fri, Jun 6, 2014 at 2:36 PM, Denis Samoilov <samoilov@gmail.com> wrote:
>
> > hi,
> > I am reading the book "Zookeeper" by Flavio Junqueira and Benjamin Reed.
> > And I am now concerned if Zookeeper right tool for our scenario:
> > configuration management. We have ~2000 servers that expected to
> subscribe
> > to znode change notification: current version number. As version number
> > changed all clients will read new value and read configuration
> > correspoinding this value:
> >
> > / currentVersion "v3"
> > /versions
> >   /v1 {server1, server2, server3}
> >   /v2 {server1, server2, server5}
> >   /v3 {server0, server2, server3}
> >
> > the idea we want to update configuration within seconds (<5s)
> >
> > Is 2000 watch on same znode and than two simultaneous 2000 reads (one for
> > version and one for content) Ok for ZooKeeper?
> >
> > according the book:
> > "...One issue to be aware of is that ZooKeeper triggers all watches set
> for
> > a particular znode change when the change occurs. If there are 1,000
> > clients that have set a watch on a given znode with a call to exists,
> then
> > 1,000 notifications will be sent out when the znode is created. A change
> to
> > a watched znode might consequently generate a spike of notifica‚Äź tions.
> > Such a spike could affect, for example, the latency of operations
> submitted
> > around the time of the spike. When possible, we recommend avoiding such a
> > use of ZooKeeper in which a large number of clients watch for a change
> to a
> > given znode. It is much better to have only a few clients watching any
> > given znode at a time, and ideally at most one..."
> >
> > 1 vs 2000 is too big difference. And books says that even 1000 is a
> > problem. On other hand Zynga says that they did similar to our solution:
> >
> >
> http://code.zynga.com/2011/08/updating-thousands-of-configuration-files-in-under-a-second/
> >
> > Thank you,
> > Denis
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message