zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Nickerson <davidnickerson4mailingli...@gmail.com>
Subject Re: Memory leak caused by a watch that will never be triggered
Date Wed, 30 May 2012 19:11:10 GMT
I'm using Java.

The problem is that the zookeeper client holds onto a reference to the
watcher object. Even if that watcher object is a WeakReference object, the
zookeeper client will hold onto it indefinitely.

I've already made the watcher object very light-weight. (It has no instance
fields.) With a very large number of threads running for a very long time,
I can end up with a whole bunch of watcher objects that the zookeeper
client references.

On Wed, May 30, 2012 at 2:56 PM, Mark Gius <mgius7096@gmail.com> wrote:

> What language are you using?  Does it have some equivalent of Python's
> http://docs.python.org/library/weakref.html?  If so you can pass this weak
> reference to the watcher (and have some basic code failing gracefully if
> the referred object has been purged already).
> Mark
> On Wed, May 30, 2012 at 11:35 AM, David Nickerson <
> davidnickerson4mailinglists@gmail.com> wrote:
> > I writing a lock implementation that agrees with the lock recipe in the
> > documentation<
> > http://zookeeper.apache.org/doc/current/recipes.html#sc_recipes_Locks>.
> > In step 5, the thread needs to wait for a notification that it may have
> the
> > lock. Since I'm working with many threads, each thread needs to wait on
> its
> > own object. I decided that it would be easiest if the thread waits on the
> > watcher that is passed as a parameter to zk.exists in step 4. This way,
> > when a thread may have the lock, its watcher gets called directly.
> >
> > (The other option would be to have only one watcher. Then, when
> > watcher.process() gets called, the watcher would need to figure out which
> > thread it should wake up.)
> >
> > The only problem is that I might have a memory leak. In step 5, if
> exists()
> > returns false, then the node does not exist and it will never exist
> because
> > it is sequential. This means that the client will forever hold a watch
> for
> > a node that will never be created, and that watch will forever reference
> my
> > watcher object. Therefore, even if I try to dump that watcher object
> (since
> > I don't need it anymore), the client will still hold a reference to it
> and
> > it won't be garbage collected.
> >
> > The only way that I've come up with to plug the memory leak is to
> recreate
> > the znode as non-sequential, thereby triggering the watch I set, and then
> > delete the znode to prevent a deadlock.
> >
> > Can anyone confirm that this is indeed a memory leak, and does anyone
> know
> > a better way to prevent it?
> >

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message