Return-Path: X-Original-To: apmail-curator-user-archive@minotaur.apache.org Delivered-To: apmail-curator-user-archive@minotaur.apache.org Received: from mail.apache.org (hermes.apache.org [140.211.11.3]) by minotaur.apache.org (Postfix) with SMTP id E323F10420 for ; Tue, 29 Apr 2014 16:43:39 +0000 (UTC) Received: (qmail 11679 invoked by uid 500); 29 Apr 2014 16:43:37 -0000 Delivered-To: apmail-curator-user-archive@curator.apache.org Received: (qmail 11604 invoked by uid 500); 29 Apr 2014 16:43:37 -0000 Mailing-List: contact user-help@curator.apache.org; run by ezmlm Precedence: bulk List-Help: List-Unsubscribe: List-Post: List-Id: Reply-To: user@curator.apache.org Delivered-To: mailing list user@curator.apache.org Received: (qmail 11596 invoked by uid 99); 29 Apr 2014 16:43:37 -0000 Received: from nike.apache.org (HELO nike.apache.org) (192.87.106.230) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Apr 2014 16:43:37 +0000 X-ASF-Spam-Status: No, hits=1.5 required=5.0 tests=HTML_MESSAGE,RCVD_IN_DNSWL_LOW,SPF_PASS X-Spam-Check-By: apache.org Received-SPF: pass (nike.apache.org: domain of joelittlejohn@gmail.com designates 209.85.216.53 as permitted sender) Received: from [209.85.216.53] (HELO mail-qa0-f53.google.com) (209.85.216.53) by apache.org (qpsmtpd/0.29) with ESMTP; Tue, 29 Apr 2014 16:43:33 +0000 Received: by mail-qa0-f53.google.com with SMTP id i13so452485qae.12 for ; Tue, 29 Apr 2014 09:43:10 -0700 (PDT) DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=gmail.com; s=20120113; h=mime-version:in-reply-to:references:date:message-id:subject:from:to :cc:content-type; bh=hvk96OyFCZrR3wmE1j+oeltRVFz0aRcpZdzYiqjeWtU=; b=GSOFToglX5g9dY+zkXAvsMut67YFWo0XhEuYGAaEpeivaQLaKc5BX4jRFjUtAHN2V8 ZXagLHsFBbkZI8QYz+wmpXzHqXCdv/ha3x0imP8mwn39QPdk2A6KKaPSCb7mJZ1GcJm+ 817VQJ0qNRiaeG79qrhgzWudwsXaz5Nh0po9L8WctEcMry4sm6lFtVaVyw4vpPYp9wtV Oi1t5XxpKnTw6OTv2Q5Z+Br6LYnPg2VWxQdCENH9a6FeZ7SEQI45Vd/z+btvPxZ0VQt4 nJSmFVIyta2pn0l+Cha4YIC0/lBJnrrKXRzrJg7EkYKRvX0Z2cBCQugIPMd1OdgFAJMJ 6ALQ== MIME-Version: 1.0 X-Received: by 10.140.23.7 with SMTP id 7mr821710qgo.0.1398789790502; Tue, 29 Apr 2014 09:43:10 -0700 (PDT) Received: by 10.140.87.241 with HTTP; Tue, 29 Apr 2014 09:43:10 -0700 (PDT) In-Reply-To: References: <27f3cc24.fc62.145ae0d74ea.Coremail.shuxingzhang1988@163.com> Date: Tue, 29 Apr 2014 17:43:10 +0100 Message-ID: Subject: Re: Memory leak when using service providers From: Joe Littlejohn To: Jordan Zimmerman Cc: user@curator.apache.org Content-Type: multipart/alternative; boundary=001a11c13db294169a04f831207b X-Virus-Checked: Checked by ClamAV on apache.org --001a11c13db294169a04f831207b Content-Type: text/plain; charset=UTF-8 Thanks Jordan. I thought you might want to have a look over the problem before raising a ticket but I've added the details here: https://issues.apache.org/jira/browse/CURATOR-105 Cheers On 29 April 2014 17:24, Jordan Zimmerman wrote: > If this is a real issue can someone please open a Jira for it? > > -Jordan > > > From: Joe Littlejohn joelittlejohn@gmail.com > Reply: user@curator.apache.org user@curator.apache.org > Date: April 29, 2014 at 10:54:41 AM > To: user@curator.apache.org user@curator.apache.org > Subject: Re: Memory leak when using service providers > > References to the ServiceDiscovery instance will not stop GC, only > references in the other direction would stop GC. The problem here is some > cached values that are not cleaned up when a provider is closed, and the > caches maintain references that stop the ServiceProvider instances from > being cleaned up correctly. > > When a provider is closed, the service discovery is notified (see > org.apache.curator.x.discovery.details.ServiceProviderImpl#close). This > allows the ServiceDiscovery to remove that provider from the list of > providers that it will close when the ServiceDiscovery itself is closed. > > > On 29 April 2014 16:14, shuxingzhang1988 wrote: > >> Hi >> >> I think it maybe not a problem. In the test code,the so many >> serviceprovders have the reference to the first servicediscovery which is >> not released a,so they can not gc . >> >> >> >> >> >> At 2014-04-29 18:55:55,"Joe Littlejohn" wrote: >> >> Hi, >> >> I've observed a memory leak in our production system using Curator >> service discovery. I can replicate the problem with 2.4.1 and 2.4.2, though >> I haven't tested any older versions. >> >> This test shows the problem: >> >> https://www.refheap.com/82891 >> >> (Hopefully this test is enough to demonstrate the problem. I haven't used >> the TestingServer so you'll need a zk instance with a registered service.) >> >> If you run it and watch the process with jvisualvm you'll see that the >> heap grows and grows as the test is running. Taking a heap dump will reveal >> thousands of ServiceInstance and ServiceCacheImpl instances that are >> retained even though the provider is closed after each usage. The >> references appear to be traced back to the PathChildrenCache. I realise >> it's possible to retain ServiceProvider instances, but this appears to be a >> leak that shouldn't occur if the provider is correctly closed each time. >> >> Can anyone comment on whether this is indeed a problem? Am I missing >> something? >> >> Cheers >> >> >> >> > --001a11c13db294169a04f831207b Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable
Thanks Jordan. I thought you might want to have a look ove= r the problem before raising a ticket but I've added the details here:<= div>

Cheer= s


On 29 = April 2014 17:24, Jordan Zimmerman <jordan@jordanzimmerman.com> wrote:
If this is a real issue can someone please open a Jira for it?

-Jordan


From:=C2=A0Joe Littlejohn joelittlej= ohn@gmail.com
Reply:=C2=A0user@curator.apache.org user@curator.apache.org
Date:=C2=A0= April 29, 2014 at 10:54:41 AM
To:=C2=A0user@curator.apache.org user@curator.apache.org
Subject:=C2=A0 Re: Memory leak when using service providers

References to the ServiceDiscovery instance will not stop GC, only references in the other direction would stop GC. The problem here is some cached values that are not cleaned up when a provider is closed, and the caches maintain references that stop the ServiceProvider instances from being cleaned up correctly.

When a provider is closed, the service discovery is notified (see org.apache.curator.x.discovery.details.ServiceProviderImpl#close). This allows the ServiceDiscovery to remove that provider from the list of providers that it will close when the ServiceDiscovery itself is closed.


On 29 April 2014 16:14, shuxingzhang1988 <shuxingzhang1988@163.com> wrote:
Hi

I think it maybe not a problem. In the test code,the so many serviceprovders=C2=A0 have the reference to the first servicediscovery=C2=A0 which is not released a,so they can not gc=C2=A0 .





At 2014-04-29 18:55:55,"Joe=C2=A0Littlejohn"=C2=A0<joelittlejohn@gmail.com> wrote:
Hi,

I've observed a memory leak in our production system using Curator service discovery. I can replicate the problem with 2.4.1 and 2.4.2, though I haven't tested any older versions.=C2=A0

This test shows the problem:


(Hopefully this test is enough to demonstrate the problem. I haven't used the TestingServer so you'll need a zk instance with a registered service.)

If you run it and watch the process with jvisualvm you'll see that the heap grows and grows as the test is running. Taking a heap dump will reveal thousands of ServiceInstance and ServiceCacheImpl instances that are retained even though the provider is closed after each usage. The references appear to be traced back to the PathChildrenCache. I realise it's possible to retain ServiceProvider instances, but this appears to be a leak that shouldn't occur if the provider is correctly closed each time.

Can anyone comment on whether this is indeed a problem? Am I missing something?

Cheers




--001a11c13db294169a04f831207b--