zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Martin Waite <waite....@googlemail.com>
Subject Re: Managing multi-site clusters with Zookeeper
Date Mon, 08 Mar 2010 13:40:10 GMT
Hi Mahadev,

Thanks again for your insight.

I will no doubt be in touch to let you know how this works out.

regards,
Martin

On 7 March 2010 23:00, Mahadev Konar <mahadev@yahoo-inc.com> wrote:

> Martin,
>  2Mb link might certainly be a problem. We can refer to these nodes as
> ZooKeeper servers. Znodes is used to data elements in the ZooKeeper data
> tree.
>
> The Zookeeper ensemble has minimal traffic which is basically health checks
> between the members of the ensemble. We call one of the members as Leader
> who is leading the ensemble and the others as Followers. The Leader does
> periodic health checks to see if the Followers are doing fine. This is of
> the order of << 1KB/sec.
>
> There is some traffic when the leader election within the ensemble happens.
> This might be of the order of 1-2KB/sec.
>
> As you mentioned the reads happen locally. So, a good enough link within
> the
> ensemble members is important so that these followers can be up to date
> with
> the Leader. But again looking at your config, looks like its mostly read
> only traffic.
>
> One more thing you should be aware of:
> Lets says a ephemeral node was created and the client died, then the
> clients
> connected to the slow ZooKeeper server (with 2Mb/s links) would lag behind
> the other clients connected to the other servers.
>
> As per my opinion you should do some testing since 2Mb/sec seems a little
> dodgy.
>
> Thanks
> mahadev
>
> On 3/7/10 2:09 PM, "Martin Waite" <waite.134@googlemail.com> wrote:
>
> > Hi Mahadev,
> >
> > The inter-site links are a nuisance.  We have two data-centres with 100Mb
> > links which I hope would be good enough for most uses, but we need a 3rd
> > site - and currently that only has 2Mb links to the other sites.  This
> might
> > be a problem.
> >
> > The ensemble would have a lot of read traffic from applications asking
> which
> > database to connect to for each transaction - which presumably would be
> > mostly handled by local zookeeper servers (do we call these "nodes" as
> > opposed to znodes ?).  The write traffic would be mostly changes to
> > configuration (a rare event), and changes in the health of database
> servers
> > - also hopefully rare.  I suppose the main concern is how much ambient
> > zookeeper system chatter will cross the links.   Are there any
> measurements
> > of how much traffic is used by zookeeper in maintaining the ensemble ?
> >
> > Another question that occurs is whether I can link sites A,B, and C in a
> > ring - so that if any one site drops out, the remaining 2 continue to
> talk.
> > I suppose that if the zookeeper servers are all in direct contact with
> each
> > other, this issue does not exist.
> >
> > regards,
> > Martin
> >
> > On 7 March 2010 21:43, Mahadev Konar <mahadev@yahoo-inc.com> wrote:
> >
> >> Hi Martin,
> >>  As Ted rightly mentions that ZooKeeper usually is run within a colo
> >> because
> >> of the low latency requirements of applications that it supports.
> >>
> >> Its definitely reasnoble to use it in a multi data center environments
> but
> >> you should realize the implications of it. The high latency/low
> throughput
> >> means that you should make minimal use of such a ZooKeeper ensemble.
> >>
> >> Also, there are things like the tick Time, the syncLimit and others
> (setup
> >> parameters for ZooKeeper in config) which you will need to tune a little
> to
> >> get ZooKeeper running without many hiccups in this environment.
> >>
> >> Thanks
> >> mahadev
> >>
> >>
> >> On 3/6/10 10:29 AM, "Ted Dunning" <ted.dunning@gmail.com> wrote:
> >>
> >>> What you describe is relatively reasonable, even though Zookeeper is
> not
> >>> normally distributed across multiple data centers with all members
> >> getting
> >>> full votes.  If you account for the limited throughput that this will
> >> impose
> >>> on your applications that use ZK, then I think that this can work well.
> >>> Probably, you would have local ZK clusters for higher transaction rate
> >>> applications.
> >>>
> >>> You should also consider very carefully whether having multiple data
> >> centers
> >>> increases or decreases your overall reliability.  Unless you design
> very
> >>> carefully, this will normally substantially degrade reliability.
>  Making
> >>> sure that it increases reliability is a really big task that involves a
> >> lot
> >>> of surprising (it was to me) considerations and considerable hardware
> and
> >>> time investments.
> >>>
> >>> Good luck!
> >>>
> >>> On Sat, Mar 6, 2010 at 1:50 AM, Martin Waite <waite.134@googlemail.com
> >>> wrote:
> >>>
> >>>> Is this a viable approach, or am I taking Zookeeper out of its
> >> application
> >>>> domain and just asking for trouble ?
> >>>>
> >>>
> >>>
> >>
> >>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message