zookeeper-user mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From David Nickerson <davidnickerson4mailingli...@gmail.com>
Subject Re: Memory leak caused by a watch that will never be triggered
Date Fri, 15 Jun 2012 14:05:33 GMT
I found a better solution. By using getData() instead of exists(), a watch
won't be set if the node doesn't exist.

On Fri, Jun 1, 2012 at 11:36 AM, David Nickerson <
davidnickerson4mailinglists@gmail.com> wrote:

> The solution to my problem is this pending feature<https://issues.apache.org/jira/browse/ZOOKEEPER-442>
> .
>
>
> On Wed, May 30, 2012 at 3:11 PM, David Nickerson <
> davidnickerson4mailinglists@gmail.com> wrote:
>
>> I'm using Java.
>>
>> The problem is that the zookeeper client holds onto a reference to the
>> watcher object. Even if that watcher object is a WeakReference object, the
>> zookeeper client will hold onto it indefinitely.
>>
>> I've already made the watcher object very light-weight. (It has no
>> instance fields.) With a very large number of threads running for a very
>> long time, I can end up with a whole bunch of watcher objects that
>> the zookeeper client references.
>>
>>
>> On Wed, May 30, 2012 at 2:56 PM, Mark Gius <mgius7096@gmail.com> wrote:
>>
>>> What language are you using?  Does it have some equivalent of Python's
>>> http://docs.python.org/library/weakref.html?  If so you can pass this
>>> weak
>>> reference to the watcher (and have some basic code failing gracefully if
>>> the referred object has been purged already).
>>>
>>> Mark
>>>
>>> On Wed, May 30, 2012 at 11:35 AM, David Nickerson <
>>> davidnickerson4mailinglists@gmail.com> wrote:
>>>
>>> > I writing a lock implementation that agrees with the lock recipe in the
>>> > documentation<
>>> > http://zookeeper.apache.org/doc/current/recipes.html#sc_recipes_Locks
>>> >.
>>> > In step 5, the thread needs to wait for a notification that it may
>>> have the
>>> > lock. Since I'm working with many threads, each thread needs to wait
>>> on its
>>> > own object. I decided that it would be easiest if the thread waits on
>>> the
>>> > watcher that is passed as a parameter to zk.exists in step 4. This way,
>>> > when a thread may have the lock, its watcher gets called directly.
>>> >
>>> > (The other option would be to have only one watcher. Then, when
>>> > watcher.process() gets called, the watcher would need to figure out
>>> which
>>> > thread it should wake up.)
>>> >
>>> > The only problem is that I might have a memory leak. In step 5, if
>>> exists()
>>> > returns false, then the node does not exist and it will never exist
>>> because
>>> > it is sequential. This means that the client will forever hold a watch
>>> for
>>> > a node that will never be created, and that watch will forever
>>> reference my
>>> > watcher object. Therefore, even if I try to dump that watcher object
>>> (since
>>> > I don't need it anymore), the client will still hold a reference to it
>>> and
>>> > it won't be garbage collected.
>>> >
>>> > The only way that I've come up with to plug the memory leak is to
>>> recreate
>>> > the znode as non-sequential, thereby triggering the watch I set, and
>>> then
>>> > delete the znode to prevent a deadlock.
>>> >
>>> > Can anyone confirm that this is indeed a memory leak, and does anyone
>>> know
>>> > a better way to prevent it?
>>> >
>>>
>>
>>
>

Mime
  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message