zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Ted Dunning <ted.dunn...@gmail.com>
Subject Re: Question about sharing Zookeeper connections
Date Fri, 18 Feb 2011 19:40:39 GMT
Most applications just share a single connection for all uses of a single
class.  That might be viewed at the logically simplest extreme of a simple
connection pool.

Why did you need more than one connection for each kind of use?

On Fri, Feb 18, 2011 at 11:13 AM, Peco Karayanev <pecomeister@gmail.com>wrote:

> Hi,
> I slightly disagree with the priority for a "connection pool" for
> zookeeper.
> I had to implement a connection pool/reuse for a smaller environment (under
> 50 nodes). I have a toolchain that can be composed of parts, and each part
> needed some synchronization through zookeeper. So even on a smaller system
> these heavier toolchains started using a lot of physical sessions and
> connections. Hence the connection pool. Also from performance perspective
> establishing a TCP connection for every session adds latency overhead.
>
> Cheers
> Peco
>
>
> On Fri, Feb 18, 2011 at 11:58 AM, Benjamin Reed <ben.reed@gmail.com>
> wrote:
>
> > it does seem like a good idea to make multiple zk handles share a
> > connection, but as ted points out, they may have different timeouts,
> which
> > would make the sharing logic quite complicated. i think the
> implementation
> > might also be quite complicated. having said that, if someone could come
> up
> > with a simple and correct connection sharing implementation, we (or at
> > least
> > i) would be open to it.
> >
> > ben
> >
> > On Fri, Feb 18, 2011 at 9:31 AM, Ted Dunning <ted.dunning@gmail.com>
> > wrote:
> >
> > > On Fri, Feb 18, 2011 at 1:24 AM, YUNG-LIN HO <yunglin@gmail.com>
> wrote:
> > >
> > > > Because zookeeper clients will try to keep session alive by sending a
> > > ping
> > > > request every 2 seconds. If libraries in an application do not share
> > > > connections with each other, they would flood the zookeeper server
> with
> > > > unnecessary requests and drag down performance of the server.
> > > >
> > >
> > > Make sure that you have a valid reason to worry first.
> > >
> > > Do you have thousands of clients?
> > >
> > > If not, these keep-alives are likely to be undetectable, load-wise.
> > >
> > >
> > > > I am wondering is there any connection manager exists in the
> > > > Hadoop/Zookeeper project that helps users to share connections?
> > > >
> > >
> > > Yes.  Zookeeper.
> > >
> > > Just open a single connection and pass it around via a singleton of
> some
> > > kind or your favorite dependency injection technique.
> > >
> > > This isn't always a great idea since your disconnect and expiration
> > > strategies might differ between different uses in important ways.
> > >
> >
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message