zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Nickerson <davidnickerson4mailingli...@gmail.com>
Subject Re: Memory leak caused by a watch that will never be triggered
Date Fri, 01 Jun 2012 15:36:58 GMT
The solution to my problem is this pending
feature<https://issues.apache.org/jira/browse/ZOOKEEPER-442>
.

On Wed, May 30, 2012 at 3:11 PM, David Nickerson <
davidnickerson4mailinglists@gmail.com> wrote:

> I'm using Java.
>
> The problem is that the zookeeper client holds onto a reference to the
> watcher object. Even if that watcher object is a WeakReference object, the
> zookeeper client will hold onto it indefinitely.
>
> I've already made the watcher object very light-weight. (It has no
> instance fields.) With a very large number of threads running for a very
> long time, I can end up with a whole bunch of watcher objects that
> the zookeeper client references.
>
>
> On Wed, May 30, 2012 at 2:56 PM, Mark Gius <mgius7096@gmail.com> wrote:
>
>> What language are you using?  Does it have some equivalent of Python's
>> http://docs.python.org/library/weakref.html?  If so you can pass this
>> weak
>> reference to the watcher (and have some basic code failing gracefully if
>> the referred object has been purged already).
>>
>> Mark
>>
>> On Wed, May 30, 2012 at 11:35 AM, David Nickerson <
>> davidnickerson4mailinglists@gmail.com> wrote:
>>
>> > I writing a lock implementation that agrees with the lock recipe in the
>> > documentation<
>> > http://zookeeper.apache.org/doc/current/recipes.html#sc_recipes_Locks>.
>> > In step 5, the thread needs to wait for a notification that it may have
>> the
>> > lock. Since I'm working with many threads, each thread needs to wait on
>> its
>> > own object. I decided that it would be easiest if the thread waits on
>> the
>> > watcher that is passed as a parameter to zk.exists in step 4. This way,
>> > when a thread may have the lock, its watcher gets called directly.
>> >
>> > (The other option would be to have only one watcher. Then, when
>> > watcher.process() gets called, the watcher would need to figure out
>> which
>> > thread it should wake up.)
>> >
>> > The only problem is that I might have a memory leak. In step 5, if
>> exists()
>> > returns false, then the node does not exist and it will never exist
>> because
>> > it is sequential. This means that the client will forever hold a watch
>> for
>> > a node that will never be created, and that watch will forever
>> reference my
>> > watcher object. Therefore, even if I try to dump that watcher object
>> (since
>> > I don't need it anymore), the client will still hold a reference to it
>> and
>> > it won't be garbage collected.
>> >
>> > The only way that I've come up with to plug the memory leak is to
>> recreate
>> > the znode as non-sequential, thereby triggering the watch I set, and
>> then
>> > delete the znode to prevent a deadlock.
>> >
>> > Can anyone confirm that this is indeed a memory leak, and does anyone
>> know
>> > a better way to prevent it?
>> >
>>
>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message