zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Peco Karayanev <pecomeis...@gmail.com>
Subject Re: Question about sharing Zookeeper connections
Date Fri, 18 Feb 2011 19:13:22 GMT
Hi,
I slightly disagree with the priority for a "connection pool" for zookeeper.
I had to implement a connection pool/reuse for a smaller environment (under
50 nodes). I have a toolchain that can be composed of parts, and each part
needed some synchronization through zookeeper. So even on a smaller system
these heavier toolchains started using a lot of physical sessions and
connections. Hence the connection pool. Also from performance perspective
establishing a TCP connection for every session adds latency overhead.

Cheers
Peco


On Fri, Feb 18, 2011 at 11:58 AM, Benjamin Reed <ben.reed@gmail.com> wrote:

> it does seem like a good idea to make multiple zk handles share a
> connection, but as ted points out, they may have different timeouts, which
> would make the sharing logic quite complicated. i think the implementation
> might also be quite complicated. having said that, if someone could come up
> with a simple and correct connection sharing implementation, we (or at
> least
> i) would be open to it.
>
> ben
>
> On Fri, Feb 18, 2011 at 9:31 AM, Ted Dunning <ted.dunning@gmail.com>
> wrote:
>
> > On Fri, Feb 18, 2011 at 1:24 AM, YUNG-LIN HO <yunglin@gmail.com> wrote:
> >
> > > Because zookeeper clients will try to keep session alive by sending a
> > ping
> > > request every 2 seconds. If libraries in an application do not share
> > > connections with each other, they would flood the zookeeper server with
> > > unnecessary requests and drag down performance of the server.
> > >
> >
> > Make sure that you have a valid reason to worry first.
> >
> > Do you have thousands of clients?
> >
> > If not, these keep-alives are likely to be undetectable, load-wise.
> >
> >
> > > I am wondering is there any connection manager exists in the
> > > Hadoop/Zookeeper project that helps users to share connections?
> > >
> >
> > Yes.  Zookeeper.
> >
> > Just open a single connection and pass it around via a singleton of some
> > kind or your favorite dependency injection technique.
> >
> > This isn't always a great idea since your disconnect and expiration
> > strategies might differ between different uses in important ways.
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message