qpid-users mailing list archives

Site index · List index
Message view « Date » · « Thread »
Top « Date » · « Thread »
From Praveen M <lefthandma...@gmail.com>
Subject Java Broker - creating and closing too many connections causes memory leak?
Date Fri, 28 Oct 2011 19:01:53 GMT

  I ran the following test a few times and I ran into a OOM exception. Can
someone please tell me if i'm doing something wrong or if this could
potentially be a bug? Thanks.

1) Create 1000 queues.
2) Create 4 consumers -> each consumer uses it's own connection
3) In a loop do the following,

for each queues 1..1000 {
    create producer(create new connection) for queue |i|
    produce 1 message for that queue.
    close producer (close connection)

4) Wait till all messages are consumed by the 4 consumers and then close all
4 consumers.
5) Re-Run the test without a broker restart

Each time the test ran, it created and closed 1004 connections (with at most
5 connections living in parallel...we close the earlier queue's producer's
connection before creating a producer for the next queue).

Also, I checked for any stray threads on the client side due to any unclosed
connection, and it looks like everything was closed fine (I didn't see any
stray threads).

On the broker side, I saw that the memory usage creeps up each time i re-run
the test, and eventually hits a OOM. (Running the above test about 5 times
causes a OOM )

I tried profiling the broker to see what's holding on to all that memory and
saw a certain class's(org.apache.qpid.transport.Method[]) instance count
grow up each time my test was run and it was holding the maximum memory in
the broker.

The number of instances of org.apache.qpid.transport.Method[] were trending
as follows:

After first test - 2008
After second test - 4016
... and so on.

There seems to be a direct relation between the number of those objects and
the connections created and closed. ~ Why are these objects not freed on
closing the connection??
I'm using the Java Broker out of trunk and using it with a DerbyMessageStore
for the test.

Or is there something fundamentally wrong by the way I run the test, I guess
opening and closing so many connections isn't a great idea (maybe I should
be dealing with sessions). But still, I see that this can be a potential
problem in a really long running broker system(where we cannot afford to
restart the broker periodically) which can have clients joining/leaving at
regular intervals.

Thank you for all the effort answering our questions,

  • Unnamed multipart/alternative (inline, None, 0 bytes)
View raw message