Mailing-List: contact users-help@qpid.apache.org; run by ezmlm
Precedence: bulk
Reply-To: users@qpid.apache.org
Received-SPF: pass (nike.apache.org: domain of danushka.menikkumbura@gmail.com
 designates 209.85.210.170 as permitted sender)
MIME-Version: 1.0
In-Reply-To: 
 <CAEqX17PK=pUOyJfnbYm2KCv4Zb4UywMVtmxufRE2jB_r3dL=uQ@mail.gmail.com>
References: 
 <CAEqX17PK=pUOyJfnbYm2KCv4Zb4UywMVtmxufRE2jB_r3dL=uQ@mail.gmail.com>
From: Danushka Menikkumbura <danushka.menikkumbura@gmail.com>
Date: Sat, 29 Oct 2011 00:42:15 +0530
Message-ID: 
 <CAF4V=3Gs0xc9ZyzbRGN1O=RsHXrKoJ-T1dJx5v3H1x=rQ4oyEQ@mail.gmail.com>
Subject: Re: Java Broker - creating and closing too many connections causes
 memory leak?
To: users@qpid.apache.org
Content-Type: multipart/alternative; boundary=001517740adc02d9a104b060ac8c

--001517740adc02d9a104b060ac8c
Content-Type: text/plain; charset=ISO-8859-1

Hi Praveen,

AFAIK what you see is a known issue in the Java broker. I ran into the same
issue when I used the Java broker in a messaging product based on the Java
broker. Our team has submitted a couple of patches too but then the code
base was in the middle of some major refactoring. I think Robbie is the
right person to elaborate on this.

Thanks,
Danushka

On Sat, Oct 29, 2011 at 12:31 AM, Praveen M <lefthandmagic@gmail.com> wrote:

> Hi,
>
>  I ran the following test a few times and I ran into a OOM exception. Can
> someone please tell me if i'm doing something wrong or if this could
> potentially be a bug? Thanks.
>
> 1) Create 1000 queues.
> 2) Create 4 consumers -> each consumer uses it's own connection
> 3) In a loop do the following,
>
> for each queues 1..1000 {
>    create producer(create new connection) for queue |i|
>    produce 1 message for that queue.
>    close producer (close connection)
> }
>
> 4) Wait till all messages are consumed by the 4 consumers and then close
> all
> 4 consumers.
> 5) Re-Run the test without a broker restart
>
> Each time the test ran, it created and closed 1004 connections (with at
> most
> 5 connections living in parallel...we close the earlier queue's producer's
> connection before creating a producer for the next queue).
>
> Also, I checked for any stray threads on the client side due to any
> unclosed
> connection, and it looks like everything was closed fine (I didn't see any
> stray threads).
>
> On the broker side, I saw that the memory usage creeps up each time i
> re-run
> the test, and eventually hits a OOM. (Running the above test about 5 times
> causes a OOM )
>
> I tried profiling the broker to see what's holding on to all that memory
> and
> saw a certain class's(org.apache.qpid.transport.Method[]) instance count
> grow up each time my test was run and it was holding the maximum memory in
> the broker.
>
> The number of instances of org.apache.qpid.transport.Method[] were trending
> as follows:
>
> After first test - 2008
> After second test - 4016
> ... and so on.
>
> There seems to be a direct relation between the number of those objects and
> the connections created and closed. ~ Why are these objects not freed on
> closing the connection??
> I'm using the Java Broker out of trunk and using it with a
> DerbyMessageStore
> for the test.
>
> Or is there something fundamentally wrong by the way I run the test, I
> guess
> opening and closing so many connections isn't a great idea (maybe I should
> be dealing with sessions). But still, I see that this can be a potential
> problem in a really long running broker system(where we cannot afford to
> restart the broker periodically) which can have clients joining/leaving at
> regular intervals.
>
>
> Thank you for all the effort answering our questions,
> --
> -Praveen
>

--001517740adc02d9a104b060ac8c--